A Snakemake storage plugin that reads and writes from a locally mounted filesystem using rsync
Project description
snakemake-storage-plugin-fs
A Snakemake storage plugin that reads and writes from a locally mounted filesystem using rsync. This is particularly useful when running Snakemake on an NFS. Complex parallel IO patterns can slow down NFS quite substantially. The following Snakemake CLI flags allow to avoid such patterns by instructing Snakemake to copy any input to a fast local scatch disk and copying output files back to NFS at the end of a job.
snakemake --default-storage-provider fs --no-shared-fs --local-storage-prefix /local/work/$USER
with /local/work/$USER being the path to the local (non NFS) scratch dir.
Alternatively, these options can be persisted in a profile:
default-storage-provider: fs
no-shared-fs: true
local-storage-prefix: /local/work/$USER
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for snakemake_storage_plugin_fs-0.1.0.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | dfe0d9cb9ba90c1d925207a402af79a8a716b91eb60a3bcd4b7ffe84169b1b00 |
|
| MD5 | edd3522757c28ee378eb3f6fd8adf3ee |
|
| BLAKE2b-256 | f3571cc4ba8d768d095e06f3f49cf20008692f9d8ac92dacce4bf3728369c8af |
Hashes for snakemake_storage_plugin_fs-0.1.0-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 363c064381bc3da72f812832556f7e7195abbafd51e895e1436db6be069a7949 |
|
| MD5 | addf78f732844e4db4bb5bee8c3d717a |
|
| BLAKE2b-256 | 33bef3909384a0a033eefdd4b88a3931304f3229729cb70ecf7786471541b2a7 |
