Common Operations On Lots-of Sequences Tool

The cool-seq-tool provides:

• Transcript alignment data from the UTA database
• Fast access to sequence data using SeqRepo
• Liftover between assemblies (GRCh38 <--> GRCh37) from PyLiftover
• Lifting over to preferred MANE compatible transcript. See here for more information.

Installation

pip

pip install cool-seq-tool[dev,tests]

Development

Clone the repo:

git clone https://github.com/GenomicMedLab/cool-seq-tool
cd cool_seq_tool

Install Pipenv if necessary.

Install backend dependencies and enter Pipenv environment:

pipenv shell
pipenv update
pipenv install --dev

UTA Database Installation

cool-seq-tool uses intalls local UTA database. For other ways to install, visit biocommons.uta.

Local Installation

The following commands will likely need modification appropriate for the installation environment.

1. Install PostgreSQL

2. Create user and database.

   $ createuser -U postgres uta_admin
   $ createuser -U postgres anonymous
   $ createdb -U postgres -O uta_admin uta
   

3. To install locally, from the cool_seq_tool/data directory:

export UTA_VERSION=uta_20210129.pgd.gz
curl -O http://dl.biocommons.org/uta/$UTA_VERSION
gzip -cdq ${UTA_VERSION} | grep -v "^REFRESH MATERIALIZED VIEW" | psql -h localhost -U uta_admin --echo-errors --single-transaction -v ON_ERROR_STOP=1 -d uta -p 5433

UTA Installation Issues

If you have trouble installing UTA, you can visit these two READMEs.

Connecting to the database

To connect to the UTA database, you can use the default url (postgresql://uta_admin@localhost:5433/uta/uta_20210129). If you use the default url, you must either set the password using environment variable UTA_PASSWORD or setting the parameter db_pwd in the UTA class.

If you do not wish to use the default, you must set the environment variable UTA_DB_URL which has the format of driver://user:pass@host:port/database/schema.

Data Downloads

SeqRepo

cool-seq-tool relies on seqrepo, which you must download yourself.

Use the SEQREPO_ROOT_DIR environment variable to set the path of an already existing SeqRepo directory. The default is /usr/local/share/seqrepo/latest.

From the root directory:

pip install seqrepo
sudo mkdir /usr/local/share/seqrepo
sudo chown $USER /usr/local/share/seqrepo
seqrepo pull -i 2021-01-29  # Replace with latest version using `seqrepo list-remote-instances` if outdated

If you get an error similar to the one below:

PermissionError: [Error 13] Permission denied: '/usr/local/share/seqrepo/2021-01-29._fkuefgd' -> '/usr/local/share/seqrepo/2021-01-29'

You will want to do the following:
(Might not be ._fkuefgd, so replace with your error message path)

sudo mv /usr/local/share/seqrepo/2021-01-29._fkuefgd /usr/local/share/seqrepo/2021-01-29
exit

transcript_mappings.tsv

cool-seq-tool uses Ensembl BioMart to retrieve cool_seq_tool/data/transcript_mappings.tsv. We currently use Human Genes (GRCh38.p13) for the dataset and the following attributes we use are: Gene stable ID, Gene stable ID version, Transcript stable ID, Transcript stable ID version, Protein stable ID, Protein stable ID version, RefSeq match transcript (MANE Select), Gene name.

Use the TRANSCRIPT_MAPPINGS_PATH environment variable to set the path of an already existing transcript_mappings.tsv. The default is cool_seq_tool/data/transcript_mapping.tsv.

LRG_RefSeqGene

cool-seq-tool fetches the latest version of LRG_RefSeqGene if the environment variable LRG_REFSEQGENE_PATH is not set. When LRG_REFSEQGENE_PATH is set, cool-seq-tool will look at this path and expect the LRG_RefSeqGene file. This file is found can be found here.

MANE Summary Data

cool-seq-tool fetches the latest version of MANE.GRCh38.*.summary.txt.gz if the environment variable MANE_SUMMARY_PATH is not set. When MANE_SUMMARY_PATH is set, cool-seq-tool will look at this path and expect the MANE Summary Data file. This file is found can be found here.

Starting the UTA Tools Service Locally

To start the service, run the following:

uvicorn cool_seq_tool.main:app --reload

Next, view the FastAPI on your local machine: http://127.0.0.1:8000/cool_seq_tool

Init coding style tests

Code style is managed by flake8 and checked prior to commit.

We use pre-commit to run conformance tests.

This ensures:

• Check code style
• Check for added large files
• Detect AWS Credentials
• Detect Private Key

Before first commit run:

pre-commit install

Testing

From the root directory of the repository:

pytest