Skip to main content

Manage DBnomics fetchers

Project description

DBnomics fetcher ops

Manage DBnomics fetchers.

Usage

Install

pip install dbnomics-fetcher-ops

Configure a fetcher

Configure:

  • GitLab private token: use --gitlab-private-token option or GITLAB_PRIVATE_TOKEN environment variable. The private token can be a personal access token. It must have the api scope.

Run:

dbnomics-fetchers -v configure scsmich --dry-run
# If everything seems OK, remove the --dry-run flag:
dbnomics-fetchers -v configure scsmich

List fetchers

dbnomics-fetchers -v list

Run fetcher pipelines

# Replace PROVIDER_SLUG by the real value:
dbnomics-fetchers -v run --provider-slug PROVIDER_SLUG

# To run a pipeline for each fetcher:
dbnomics-fetchers -v list --slug | xargs -I {} dbnomics-fetchers -v run --provider-slug {}

Development

This repository uses Poetry.

# git clone repo or fork
cd dbnomics-fetcher-ops
poetry install
cp .env.example .env

Run commands with:

poetry run dbnomics-fetchers COMMAND

To use ipdb:

poetry shell
# Find venv dir with "which python"
ipdb3 /path/to/venv/bin/dbnomics-fetchers ...

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbnomics_fetcher_ops-0.4.13.tar.gz (15.8 kB view details)

Uploaded Source

Built Distribution

dbnomics_fetcher_ops-0.4.13-py3-none-any.whl (25.0 kB view details)

Uploaded Python 3

File details

Details for the file dbnomics_fetcher_ops-0.4.13.tar.gz.

File metadata

  • Download URL: dbnomics_fetcher_ops-0.4.13.tar.gz
  • Upload date:
  • Size: 15.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.2.2 CPython/3.10.8 Linux/5.10.0-14-amd64

File hashes

Hashes for dbnomics_fetcher_ops-0.4.13.tar.gz
Algorithm Hash digest
SHA256 e6149ab20c58b5974d85c9af65d4e86ffe9b337eeb8647ca57786034084699aa
MD5 a9eaad5deb5532881774b10142ead08c
BLAKE2b-256 396ae40bf406d792805076d1bcc64297a50d353aa28e7caa5e06e239b616374b

See more details on using hashes here.

File details

Details for the file dbnomics_fetcher_ops-0.4.13-py3-none-any.whl.

File metadata

File hashes

Hashes for dbnomics_fetcher_ops-0.4.13-py3-none-any.whl
Algorithm Hash digest
SHA256 79aefaa8ba76aea692ce2ec905cbe65f312d8e2e3ec69b928ea1ff28098c40d4
MD5 9e4717340b7dff4fc27e227616e9f168
BLAKE2b-256 78d620f31bf308ac843c5f9df7f0b699fdf2d618fa14ec868435f1efa2fea18d

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page