Human Cell Atlas Data Storage System Command Line Interface
Project description
This repository is a pip installable Command Line Interface (CLI) and Python library (API) for interacting with the Data Coordination Platform (DCP) of the Human Cell Atlas (HCA).
Currently the hca package supports interaction with the Upload Service and Data Storage Service (DSS) for services such as uploading, downloading, and querying data.
The HCA CLI is compatible with Python versions 3.5+ (we are no longer compatible with Python 2.7, and our last compatible Python 2.7 version was hca==6.4.0).
Installation
pip install hca.
Usage
Documentation on readthedocs.io:
Example CLI/API usage:
To see the list of commands you can use, type hca --help.
Configuration management
The HCA CLI supports ingesting configuration from a configurable array of sources. Each source is a JSON file. Configuration sources that follow the first source update the configuration using recursive dictionary merging. Sources are enumerated in the following order (i.e., in order of increasing priority):
Site-wide configuration source, /etc/hca/config.json
User configuration source, ~/.config/hca/config.json
Any sources listed in the colon-delimited variable HCA_CONFIG_FILE
Command line options
Array merge operators: When loading a chain of configuration sources, the HCA CLI uses recursive dictionary merging to combine the sources. Additionally, when the original config value is a list, the package supports array manipulation operators, which let you extend and modify arrays defined in underlying configurations. See https://github.com/kislyuk/tweak#array-merge-operators for a list of these operators.
Development
To develop on the CLI, first run pip install -r requirements-dev.txt. You can install your locally modified copy of the hca package by running make install in the repository root directory.
To use the command line interface with a local or test DSS, first run hca (or scripts/hca if you want to use the package in-place from the repository root directory). This will create the file ~/.config/hca/config.json, which you can modify to update the value of DSSClient.swagger_url to point to the URL of the Swagger definition served by your DSS deployment. Lastly, the CLI enforces HTTPS connection to the DSS API. If you are connecting to a local DSS, make this change in dcp-cli/hca/util/__init__.py in the SwaggerClient object:
scheme = "http"
To use the Python interface with a local or test DSS, pass the URL of the Swagger definition to the DSSClient constructor via the swagger_url parameter:
client = DSSClient(swagger_url="https://dss.example.com/v1/swagger.json")
You can also layer a minimal config file on top of the default config.json using the HCA_CONFIG_FILE environment variable, for example:
export SWAGGER_URL="https://dss.staging.data.humancellatlas.org/v1/swagger.json" jq -n .DSSClient.swagger_url=env.SWAGGER_URL > ~/.config/hca/config.staging.json export HCA_CONFIG_FILE=~/.config/hca/config.staging.json
Testing
Before you run tests, first run hca dss login. This will open a browser where you can log in to authenticate with Google. Use an email address from one of the whitelisted domains (in DSS_SUBSCRIPTION_AUTHORIZED_DOMAINS_ARRAY from here).
Then make test.
Primary CI testing is through Travis CI; there is also additional testing with the Gitlab Allspark instance that runs tests for Windows. (Note that Allspark is not open to the public, members of the Human Cell Atlas project can access the Allspark cluster using the Github account associated with the Human Cell Atlas organization on Github.) If submitting PRs that have the potential of being platform-dependent, please ensure the status of “Windows Testing” is verified before merging.
Bugs
Please report bugs, issues, feature requests, etc. in the HumanCellAtlas/dcp-cli repository on GitHub.
Security Policy
See our Security Policy.
License
Licensed under the terms of the MIT License.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file hca-7.0.1.tar.gz.
File metadata
- Download URL: hca-7.0.1.tar.gz
- Upload date:
- Size: 82.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/2.0.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/39.2.0 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 194bf09cd1131874a241852656afba4796d4ede8aafbc358608c5858002d76bf |
|
| MD5 | 249bd69f0c4f38f4d1e60456717741c1 |
|
| BLAKE2b-256 | 70c7fbcacdfdbaf989a71583b2c99adf8ccdacc02d6594ff6c8cf856221c32d9 |
Provenance
File details
Details for the file hca-7.0.1-py2.py3-none-any.whl.
File metadata
- Download URL: hca-7.0.1-py2.py3-none-any.whl
- Upload date:
- Size: 99.5 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/2.0.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/39.2.0 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 1a25f8627c096bda4ea46bfcb4687a264db1b3db10376dc685f5d25e07719ae2 |
|
| MD5 | 05216d4b77b14fd386b99e25ba3026c9 |
|
| BLAKE2b-256 | 79b8a2ba84ffcfbff85edd98f636c9b20fbd37c2020645f1f78496bb876cd422 |
