Skip to main content

Command Line Interface to upload data to the European Nucleotide Archive

Project description

Python application BioConda version Pipy version

ENA upload tool

About

The program submits experimental data and respective metadata to the European Nucleotide Archive (ENA). The metadata should be provided in separate tables corresponding to the following ENA objects:

  • STUDY
  • SAMPLE
  • EXPERIMENT
  • RUN

The program to perform the following actions:

  • add: add an object to the archive
  • modify: modify an object in the archive
  • cancel: cancel a private object and its dependent objects (under development)
  • release: release a private object immediately to the public (under development)

Tool dependencies

  • curl
  • python 2.7+ including following packages:
    • Genshi
    • lxml
    • pandas

Installation

pip install ena-upload-cli

Be aware that Windows can give problems with the curl commands executed by the script. Check that curl is available in your PATH.

Usage

usage: ENA_upload [-h] --action {add,modify,cancel,release} [--study STUDY]
                  [--sample SAMPLE] [--experiment EXPERIMENT] [--run RUN]
                  [--data [FILE [FILE ...]]] --center CENTER_NAME --webin_id
                  WEBIN_ID (--password PASSWORD | --secret SECRET) [-d]


optional arguments:
  -h, --help            show this help message and exit
  --action {add,modify,cancel,release}
                         add: add an object to the archive
                         modify: modify an object in the 
  --study STUDY         table of STUDY object
  --sample SAMPLE       table of SAMPLE object
  --experiment EXPERIMENT
                        table of EXPERIMENT object
  --run RUN             table of RUN object
  --data [FILE [FILE ...]]
                        data for submission
  --center CENTER_NAME  specific to your Webin account
  --webin_id WEBIN_ID   the usermane of your Webin account
  --password PASSWORD   the password of your Webin account
  --secret SECRET       .secret file containing the password  of your Webin account
  -d, --dev             Flag to use the dev/sandbox endpoint of ENA.
  --vir                 Flag to use the viral sample template.

Mandatory arguments: --action, --center, --webin_id, --password or --secret.

ENA Webin

A Webin can be made here if you don't have one already. The --webin_id parameter makes use of the full username looking like: Webin-XXXXX. Visit Webin online to check on your submissions or dev Webin to check on test submissions.

The password parameter

To avoid exposing you password through the terminal history, it is recommended to make use of a .secret file, containing your password on the first line. Use instead of the --password parameter in the ena-upload-cli tool the --secret parameter to specify the location of the .secret file.

dev instance

By default the submission will be done using following url to ENA: https://www.ebi.ac.uk/ena/submit/drop-box/submit/?auth=ENA

Use the --dev flag if you want to do a test submission using the tool by the sandbox dev instance of ENA: https://wwwdev.ebi.ac.uk/ena/submit/drop-box/submit/?auth=ENA. A TEST submission will be discarded within 24 hours.

The data files

Supported data

  • Read data
  • Genome Assembly
  • Transcriptome Assembly
  • Template Sequence
  • Other Analyses

Most files uploaded to the ENA FTP server need to be compressed.

More information on how ENA wants to receive the files can be found here.

Test the tool

This is still a developmental version, please run the tool in the program directory for now.

inputs:

  • metadata tables
    • examples in example_table
    • Please define actions in status column e.g. add, modify, cancel, release
    • to perform bulk submission of all objects, the aliases ids in different ENA objects should be in the association where alias ids in experiment object link all objects together
  • experimental data
    • examples in example_data

outputs:

  • In the same directory of inputs
  • metadata tables with updated info in status and other relevant columns, e.g:
    • updated status: added, modified, canceled, released
    • accession ids
    • submission date

test command: add metadata and sequence data

ena_upload --action add --center 'your_center_name' --webin_id your_id --password your_password --study example_tables/ENA_template_studies.tsv --sample example_tables/ENA_template_samples.tsv --experiment example_tables/ENA_template_experiments.tsv --run example_tables/ENA_template_runs.tsv --data example_data/*gz --dev

test command: modify a metadata

ena_upload --action modify --center 'your_center_name' --webin_id your_id --password your_password --study example_tables/ENA_template_studies-2020-05-01T1421.tsv --dev

test command .secret file

ena-upload-cli --action add --center 'your_center_name' --webin_id your_id --study example_tables/ENA_template_studies.tsv --sample example_tables/ENA_template_samples.tsv --experiment example_tables/ENA_template_experiments.tsv --run example_tables/ENA_template_runs.tsv --data example_data/*gz --dev --secret .secret

test command for viral data

ena_upload --action add --center 'your_center_name' --webin_id your_id --password your_password --study example_tables/ENA_template_studies.tsv --sample example_tables/ENA_template_samples_vir.tsv --experiment example_tables/ENA_template_experiments.tsv --run example_tables/ENA_template_runs.tsv --data example_data/*gz --dev --vir

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ena-upload-cli-0.2.1.tar.gz (40.6 kB view details)

Uploaded Source

Built Distribution

ena_upload_cli-0.2.1-py3-none-any.whl (48.2 kB view details)

Uploaded Python 3

File details

Details for the file ena-upload-cli-0.2.1.tar.gz.

File metadata

  • Download URL: ena-upload-cli-0.2.1.tar.gz
  • Upload date:
  • Size: 40.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.50.0 CPython/3.8.5

File hashes

Hashes for ena-upload-cli-0.2.1.tar.gz
Algorithm Hash digest
SHA256 ff76b84f46c5e5f468a16ebf6a16271748c668363b5342d7aba9dd6518b0d1fc
MD5 3468810f1d10212dfd62eee20bcfb1ee
BLAKE2b-256 a01505189df18812db742c84c5153b5490bee15f0bcbe7265d3a760b086277c1

See more details on using hashes here.

Provenance

File details

Details for the file ena_upload_cli-0.2.1-py3-none-any.whl.

File metadata

  • Download URL: ena_upload_cli-0.2.1-py3-none-any.whl
  • Upload date:
  • Size: 48.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.50.0 CPython/3.8.5

File hashes

Hashes for ena_upload_cli-0.2.1-py3-none-any.whl
Algorithm Hash digest
SHA256 8531feb3b472a8a74c7516a882db06162ab5e42307cff44b9a7d35fc2c7ff7f5
MD5 7eeff4cc82a2b8d3391fdb018f28c130
BLAKE2b-256 04c0b44a3169a96a6169cb768ea03eec5b5471d552c7620436681944b15a38ef

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page