SLURM utilities for Python
Project description
pyslurmutils
SLURM utilities for Python.
Demo
Get an access token on rnice (prints SLURM_JWT=<token>)
export SLURM_TOKEN=$(scontrol token lifespan=3600)
Run some example jobs
python3 scripts/example.py
Run the tests (CI or locally)
export SLURM_TOKEN=...
export SLURM_USER=...
python3 -m pytest .
Execute a python function on SLURM
High-level API
API mimics python's concurrent.futures API
from pyslurmutils.futures import SlurmExecutor
with SlurmExecutor(
url,
user_name,
token,
log_directory=log_directory,
data_directory=data_directory, # TCP when not provided
) as pool:
future = pool.submit(sum, args=([1, 1],), pre_script="conda activate myenv")
assert future.result() == 2
Low-level API
from pyslurmutils.client.pyscript import SlurmPythonJobApi
with SlurmPythonJobApi(
url,
user_name,
token,
log_directory=log_directory,
data_directory=data_directory, # TCP when not provided
) as pyapi:
job_id = pyapi.spawn(sum, args=([1, 1],), pre_script="conda activate myenv")
try:
pyapi.wait_done(job_id)
pyapi.print_stdout_stderr(job_id)
assert pyapi.get_result(job_id) == 2
finally:
pyapi.clean_job_artifacts(job_id)
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pyslurmutils-0.0.1a0.tar.gz
(13.7 kB
view details)
File details
Details for the file pyslurmutils-0.0.1a0.tar.gz.
File metadata
- Download URL: pyslurmutils-0.0.1a0.tar.gz
- Upload date:
- Size: 13.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 7db96911f6d0cb11ac33da285b838c4ca9e9e25d6138068c86cb0f51d8c2d612 |
|
| MD5 | c8cddbbd75483728257b21ec763a822b |
|
| BLAKE2b-256 | c5e85c49077af4e5aaa2b2f28e2c72a3c17be665054c4be3fcc11691a797abfa |
