A CLI tool for cataloging pilot 1 data
Project description
pilot1-tools
A Command Line tool for managing data in Globus Search as well as transferring corresponding data to and from a Globus Endpoint.
Installation
These tools are available on Conda for Python 3.6, you can install them with the following:
conda create -n pilot1-env -c conda-forge -c nickolaussaint pilot1-tools
You can see the Developer Guide Installation for more options.
Setting Your Local
The first time you run pilot, you may encounter an error an error like the one below:
RuntimeError: Click will abort further execution because Python 3 was
configured to use ASCII as encoding for the environment.
This can happen if on systems with custom configured UTF-8 settings. You should see a list of encodings your system supports, simply choose one and set it. For example:
export LC_ALL=en_US.utf-8
export LANG=en_US.utf-8
Replace en_US.utf-8 with an encoding your system supports.
Quick Start
For a full walkthrough, see the User Guide. Administrators can also view the Admin Guide.
A quick walkthrough is below.
First, login using Globus:
pilot login
Then choose your project. See pilot project info for info on any listed project:
pilot project
pilot project set <myproject>
You can use list to get a high level overview of the data:
pilot list
If you want more detail about a specific search record, you can use describe to view details:
pilot describe dose_response/rescaled_combined_single_drug_growth
You can also download the data associated with the search record:
pilot download dose_response/rescaled_combined_single_drug_growth
When you want to add more data to the collection, you can use the upload command. This will upload the data in addition to creating a record in Globus Search to track it.
touch my_data.tsv
pilot upload my_data.tsv test_dir --dry-run --verbose -j my_metadata.json
The two flags ‘–dry-run –verbose’ are optional but handy for testing. ‘-j my_metadata.json’ is for providing any extra metadata the pilot tool can’t automatically determine. Here is an example of the metadata:
{
"title": "Drug Identifiers",
"description": "Drug identifiers, including InChIKey, SMILES, and PubChem.",
"data_type": "Drug Response",
"dataframe_type": "List",
"source": [
"InChIKey",
"SMILES",
"PubChem"
]
}
Running Tests
Ensure packages in test-requirements.txt are installed, then run:
pytest
And for coverage:
pytest --cov pilot
Building for Conda
Currently, the tableschema package has not been built for python 3.7, so this only lists instructions for python 3.6. Two channels must be used, nickolaussaint and conda-forge. The nickolaussaint channel contains fair-research-login, and conda-forge contains various other packages we need including the globus-sdk.
conda build -c nickolaussaint -c conda-forge --python 3.6 .
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for globus_pilot-0.5.0.dev0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fa14d625075154d8b3d5fa7a597e30bd954af9a3ec18d0967b8b523d1e280f95 |
|
MD5 | db5577c7ec4eca1e0c143d8793beb47f |
|
BLAKE2b-256 | 386dbf6aee2f8ccf5a86729b8f6c4eda1bd14dea851985b7af8f05e2500af461 |