SLURM utilities for Python
Project description
pyslurmutils
SLURM utilities for Python.
Demo
Get an access token on rnice
export SLURM_TOKEN=$(scontrol token lifespan=3600)
export SLURM_URL=...
export SLURM_USER=...
Run some example jobs
python3 scripts/example.py
Run the tests (CI or locally)
python3 -m pytest .
When SLURM_TOKEN, SLURM_URL or SLURM_USER is missing it will mock
the SLURM clients.
Execute a python function on SLURM
High-level API
API mimics python's concurrent.futures API
from pyslurmutils.futures import SlurmExecutor
with SlurmExecutor(
url,
user_name,
token,
log_directory=log_directory, # for log files
data_directory=data_directory, # TCP when not provided
) as pool:
future = pool.submit(sum, args=([1, 1],), pre_script="conda activate myenv")
assert future.result() == 2
Low-level API
from pyslurmutils.client.pyscript import SlurmPythonJobApi
with SlurmPythonJobApi(
url,
user_name,
token,
log_directory=log_directory, # for log files
data_directory=data_directory, # TCP when not provided
) as pyapi:
future = pyapi.spawn(sum, args=([1, 1],), pre_script="conda activate myenv")
job_id = future.job_id
try:
pyapi.wait_done(job_id)
pyapi.print_stdout_stderr(job_id)
assert pyapi.get_future(job_id).result() == 2
finally:
pyapi.clean_job_artifacts(job_id)
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pyslurmutils-0.0.3a0.tar.gz
(15.9 kB
view details)
File details
Details for the file pyslurmutils-0.0.3a0.tar.gz.
File metadata
- Download URL: pyslurmutils-0.0.3a0.tar.gz
- Upload date:
- Size: 15.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 93bfafd7f6a5c755e3981d3a643942829844e4967a475068f068cc7cb1808b59 |
|
| MD5 | 49e99d052b6f36fb30a95ee24ea542a3 |
|
| BLAKE2b-256 | 567f021606b4f622e0ed4bb57463ed8852c7399aec2bade6e2fab7fec5ed225f |
