Skip to main content

Manage DBnomics fetchers

Project description

DBnomics fetcher ops

Manage DBnomics fetchers.

Usage

Install

pip install dbnomics-fetcher-ops

Configure a fetcher

Configure:

  • GitLab private token: use --gitlab-private-token option or GITLAB_PRIVATE_TOKEN environment variable. The private token can be a personal access token. It must have the api scope.

Run:

dbnomics-fetchers -v configure scsmich --dry-run
# If everything seems OK, remove the --dry-run flag:
dbnomics-fetchers -v configure scsmich

List fetchers

dbnomics-fetchers -v list

Run fetcher pipelines

# Replace PROVIDER_SLUG by the real value:
dbnomics-fetchers -v run --provider-slug PROVIDER_SLUG

# To run a pipeline for each fetcher:
dbnomics-fetchers -v list --slug | xargs -I {} dbnomics-fetchers -v run --provider-slug {}

Development

This repository uses Poetry.

# git clone repo or fork
cd dbnomics-fetcher-ops
poetry install
cp .env.example .env

Run commands with:

poetry run dbnomics-fetchers COMMAND

To use ipdb:

poetry shell
# Find venv dir with "which python"
ipdb3 /path/to/venv/bin/dbnomics-fetchers ...

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbnomics-fetcher-ops-0.4.10.tar.gz (16.2 kB view details)

Uploaded Source

Built Distribution

dbnomics_fetcher_ops-0.4.10-py3-none-any.whl (25.2 kB view details)

Uploaded Python 3

File details

Details for the file dbnomics-fetcher-ops-0.4.10.tar.gz.

File metadata

  • Download URL: dbnomics-fetcher-ops-0.4.10.tar.gz
  • Upload date:
  • Size: 16.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.1.13 CPython/3.10.5 Linux/5.10.0-14-amd64

File hashes

Hashes for dbnomics-fetcher-ops-0.4.10.tar.gz
Algorithm Hash digest
SHA256 d448927f8df30c61735a3b9dfb1e098a0bbce2038734919a09ba5e3beab4eb83
MD5 d93477152869acf5601fc0aa259cfa15
BLAKE2b-256 f58154fe5a676c8060d895a15000988fe511a0c5e3054362157411939a957192

See more details on using hashes here.

File details

Details for the file dbnomics_fetcher_ops-0.4.10-py3-none-any.whl.

File metadata

File hashes

Hashes for dbnomics_fetcher_ops-0.4.10-py3-none-any.whl
Algorithm Hash digest
SHA256 301aa1b48ccb25f05776145ef44269d079ddebb834be1df72333eb78c0fbf141
MD5 3ea4315a01e73666b909ed6501c9cf02
BLAKE2b-256 58811ded81391abd97456f716069eb183ad05ee13d47ff746f72014c4ff6b104

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page