Export metadata from the DCP Sheepdog API.
Project description
sheepdog-exporter 🐑🐑
====================
Export metadata from the DCP!
First, download your credentials from the DCP Web interface.
Then, install sheepdog exporter: ``pip install sheepdog-exporter``.
You can now find programs to export using: ``sheepdog-exporter``!
Once you've found a program and project you want to export, use:
``sheepdog-exporter my_program my_project``. This will create a JSON
file in the current directory with all of the relevant metadata!
You can change the output path, path to your credentials, or the DCP URL
using arguments:
::
sheepdog-exporter program project --dcp-url my-url --credentials path/to/credentials --output-path /path/to/write/output
The resulting JSON has the form:
::
{
"data_objects": [{DataObject},...],
"metadata": {
metadata_type: [{metadata_value}], ...
}
}
Metadata types that are expected to have files associated with them will
have an ``object_id`` that matches an ``object_id`` in the data object
manifest.
Development
-----------
- A simple test demonstrates usage of the Exporter class in ``test``.
Using the Python API
~~~~~~~~~~~~~~~~~~~~
The exporter is available for reuse from your Python code, instantiate
it with the location of the credentials.json and the location of the DCP
instance you would like to export from:
::
from sheepdog_exporter import Exporter
exporter = Exporter('path/to/credentials', 'https://dcp.bionimbus.org')
my_export = exporter.export('topmed', 'public')
Other convenience methods are available for listing programs and
projects: ``.get_projects()`` and ``.get_programs()``.
See `sheepdog\_exporter.py <sheepdog_exporter.py>`__ for more.
Releases
~~~~~~~~
Releases are made on the creation of a new tag. Versioning follows
semantic versioning practices. Any breaking changes to the API should
result in a change in the major version.
To change the release maintainer, edit the ``travis.yml`` to remove the
pypi release details. Then follow the directions on `encrypting your
PyPi credentials using Travis
CLI <https://docs.travis-ci.com/user/deployment/pypi/>`__.
Issues
------
- Provenance to the original JSON schemas are lost. #9
- It has not been tested if JSON dumps can be used as sheepdog input.
- Some functions in the exporter are unused.
- Print messages cannot be easily suppressed. #12
- Errors in converting data from DCP are not always relayed properly to
the CLI. #11
====================
Export metadata from the DCP!
First, download your credentials from the DCP Web interface.
Then, install sheepdog exporter: ``pip install sheepdog-exporter``.
You can now find programs to export using: ``sheepdog-exporter``!
Once you've found a program and project you want to export, use:
``sheepdog-exporter my_program my_project``. This will create a JSON
file in the current directory with all of the relevant metadata!
You can change the output path, path to your credentials, or the DCP URL
using arguments:
::
sheepdog-exporter program project --dcp-url my-url --credentials path/to/credentials --output-path /path/to/write/output
The resulting JSON has the form:
::
{
"data_objects": [{DataObject},...],
"metadata": {
metadata_type: [{metadata_value}], ...
}
}
Metadata types that are expected to have files associated with them will
have an ``object_id`` that matches an ``object_id`` in the data object
manifest.
Development
-----------
- A simple test demonstrates usage of the Exporter class in ``test``.
Using the Python API
~~~~~~~~~~~~~~~~~~~~
The exporter is available for reuse from your Python code, instantiate
it with the location of the credentials.json and the location of the DCP
instance you would like to export from:
::
from sheepdog_exporter import Exporter
exporter = Exporter('path/to/credentials', 'https://dcp.bionimbus.org')
my_export = exporter.export('topmed', 'public')
Other convenience methods are available for listing programs and
projects: ``.get_projects()`` and ``.get_programs()``.
See `sheepdog\_exporter.py <sheepdog_exporter.py>`__ for more.
Releases
~~~~~~~~
Releases are made on the creation of a new tag. Versioning follows
semantic versioning practices. Any breaking changes to the API should
result in a change in the major version.
To change the release maintainer, edit the ``travis.yml`` to remove the
pypi release details. Then follow the directions on `encrypting your
PyPi credentials using Travis
CLI <https://docs.travis-ci.com/user/deployment/pypi/>`__.
Issues
------
- Provenance to the original JSON schemas are lost. #9
- It has not been tested if JSON dumps can be used as sheepdog input.
- Some functions in the exporter are unused.
- Print messages cannot be easily suppressed. #12
- Errors in converting data from DCP are not always relayed properly to
the CLI. #11
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
sheepdog-exporter-0.3.1.tar.gz
(11.7 kB
view details)
File details
Details for the file sheepdog-exporter-0.3.1.tar.gz
.
File metadata
- Download URL: sheepdog-exporter-0.3.1.tar.gz
- Upload date:
- Size: 11.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | f178a86d0882c14f8a34e6b3cb3298d95ecf84cc8afe8797a7dd4b42b8245ce6 |
|
MD5 | 35d541db858705aa64b0b1b35135c652 |
|
BLAKE2b-256 | 08a817fcd0b37895789dc362654b626cccbd6539ccacc70e7a39fa95229fdbce |