Skip to main content

Access nextflow variables from Python

Project description

This is a proof-of-concept how to make it easier to access nextflow vars from python scripts or notebooks.

Installation

git clone git@github.com:grst/nxfvars.git
cd nxfvars
pip install .

PyPI and conda will follow at some point.

Usage

Add the nxfvars.nf script to your pipeline. Import the nxfVars function as follows:

import { nxfVars } from "./nxfvars.nf"

In each process where you want to use nxfvars, add the following line to the script section, right before executing the python script or notebook:

${nxfVars(task)}

For instance:

process TEST {
    cpus 2

    input:
    val bar
    path foo
    path nxfvar

    script:
    """
    ${nxfVars(task)}
    jupyter nbconvert notebook.ipynb --execute --to html
    """
}

In your python script, add

from nxfvars import nxf

You can now retrieve inputs, params and task variables as follows:

nxf.input('bar')
nxf.task('cpus')
nxf.params('some_param')

To execute a notebook interactively (i.e. outside of nextflow), you may define default values:

nxf.input('bar', "/path/to/some/file.foo")

For a full example pipeline, see the example directory.

How it works

The groovy library serializes the nextflow variables and encodes them into an environment variable, which is injected using ${nxfVars(task)}. See also my blog post about Nextflow’s implicit variables this and task.

The nxfvars Python library decodes the env var and makes the values conveniently accessible

Ideas for the future

  • This can obviously be easily ported to other languages, e.g. R.

  • The library could be extended to serve as a minimal argument parser, i.e. that a Python script containing nxf.input("foo") could also be executed (outside Nextflow) using scripyt.py --foo=bar.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

nxfvars-0.1.0.tar.gz (97.5 kB view details)

Uploaded Source

Built Distribution

nxfvars-0.1.0-py3-none-any.whl (5.0 kB view details)

Uploaded Python 3

File details

Details for the file nxfvars-0.1.0.tar.gz.

File metadata

  • Download URL: nxfvars-0.1.0.tar.gz
  • Upload date:
  • Size: 97.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: python-requests/2.25.0

File hashes

Hashes for nxfvars-0.1.0.tar.gz
Algorithm Hash digest
SHA256 86e28c785309cd8328b4e68eefd52b87f391c1c673308abb5fa3e57e3e2dc116
MD5 bb7f621260117abdb822aa5c2c284bbf
BLAKE2b-256 33f6bdc501ef8370b24409d300dcf84985ace5b39c3e00a56f23c7ca725cb9ac

See more details on using hashes here.

Provenance

File details

Details for the file nxfvars-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: nxfvars-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 5.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: python-requests/2.25.0

File hashes

Hashes for nxfvars-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 b8fbb2f7ce749a8d983d8a77a0068db10cc8356460803a5f1294624416fabcd9
MD5 1cc78bb1a04243000f475cc1a6658a9f
BLAKE2b-256 e66103bb5c1839fac630aa6f7a6eef525a70f67e7e781a084d2b8a330fe8fe01

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page