Skip to main content

Manage DBnomics fetchers

Project description

DBnomics fetcher ops

Manage DBnomics fetchers.

Usage

Install

pip install dbnomics-fetcher-ops

Configure a fetcher

Configure:

  • GitLab private token: use --gitlab-private-token option or GITLAB_PRIVATE_TOKEN environment variable. The private token can be a personal access token. It must have the api scope.

Run:

dbnomics-fetchers -v configure scsmich --dry-run
# If everything seems OK, remove the --dry-run flag:
dbnomics-fetchers -v configure scsmich

List fetchers

dbnomics-fetchers -v list

Run fetcher pipelines

# Replace PROVIDER_SLUG by the real value:
dbnomics-fetchers -v run --provider-slug PROVIDER_SLUG

# To run a pipeline for each fetcher:
dbnomics-fetchers -v list --slug | xargs -I {} dbnomics-fetchers -v run --provider-slug {}

Development

This repository uses Poetry.

# git clone repo or fork
cd dbnomics-fetcher-ops
poetry install
cp .env.example .env

Run commands with:

poetry run dbnomics-fetchers COMMAND

To use ipdb:

poetry shell
# Find venv dir with "which python"
ipdb3 /path/to/venv/bin/dbnomics-fetchers ...

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbnomics_fetcher_ops-0.4.12.tar.gz (15.7 kB view details)

Uploaded Source

Built Distribution

dbnomics_fetcher_ops-0.4.12-py3-none-any.whl (24.9 kB view details)

Uploaded Python 3

File details

Details for the file dbnomics_fetcher_ops-0.4.12.tar.gz.

File metadata

  • Download URL: dbnomics_fetcher_ops-0.4.12.tar.gz
  • Upload date:
  • Size: 15.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.2.2 CPython/3.10.8 Linux/5.10.0-14-amd64

File hashes

Hashes for dbnomics_fetcher_ops-0.4.12.tar.gz
Algorithm Hash digest
SHA256 7734301acfbf0021152776ad1b5b5a01a461c8a17f70ba5f398d3c26c63171c8
MD5 1a7e8493a50f6cdddca0b29ad6ff9b99
BLAKE2b-256 d51eda5302da9c9b7fcc459e0d43aa1eae16b4975d84528ab0c8ba7ae9ed9bdd

See more details on using hashes here.

File details

Details for the file dbnomics_fetcher_ops-0.4.12-py3-none-any.whl.

File metadata

File hashes

Hashes for dbnomics_fetcher_ops-0.4.12-py3-none-any.whl
Algorithm Hash digest
SHA256 6bee7bc8c9031ca7500579cde9fb2dad9d16e74de11a26d032b74ead05fcdd6e
MD5 1bf9a2ba6a44cacd1426d6a5f1b4eebb
BLAKE2b-256 e4eba6e5aa71fae012c8194e2a7776436190522b4f6b338720f9bbb662ea0a53

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page