No project description provided
Project description
Snakemake Executor Kueue
This is a snakemake executor plugin that enables interaction with Kueue. The plugin will install Python dependencies that are needed, and it's assumed that you have installed Kueue and have queues configured.
under development note that the base container is a custom build under my namespace (vanessa
)
that has clones from main branches (as opposed to releases).
Usage
Setup
You will need to create a cluster first. For local development we recommend kind:
kind create cluster
Install Kueue
You will then need to install Kueue and create your local queues with cluster quotas
E.g., here is an example:
VERSION=v0.4.0
kubectl apply -f https://github.com/kubernetes-sigs/kueue/releases/download/$VERSION/manifests.yaml
Then (wait a few minutes until the jobset controller is running.) and:
kubectl apply -f example/cluster-queue.yaml
kubectl apply -f example/resource-flavor.yaml
kubectl apply -f example/user-queue.yaml
clusterqueue.kueue.x-k8s.io/cluster-queue created
resourceflavor.kueue.x-k8s.io/default-flavor created
localqueue.kueue.x-k8s.io/user-queue created
You'll also need kubernetes python installed, and of course Snakemake! Assuming you have snakemake and the plugin here installed, you should be good to go. Here is how I setup a local or development environment.
python -m venv env
source env/bin/activate
pip install .
Container
Note that while Snakemake still has a lot of moving pieces, the default container is built from the Dockerfile here and provided as vanessa/snakemake:kueue
in the executor code. Next go into an example directory to test out the Kueue executor.
Job Resources
Operator
By default, Kueue will use a batchv1/Job for each step. However, you can customize this to a different operator with the job resources via the kueue_operator attribute:
rule a:
input: ...
output: ...
resources:
kueue_operator=flux-operator
shell:
"..."
Along with a standard batchv1 Job, We currently support the following operator
s:
- flux-operator: deploy using the Flux Operator
- mpi-operator: deploy using the MPI Operator
- job: (the default) either unset, or set to "job"
Note that you are in charge of installing and configuring the various operators on your cluster! See the Kueue tasks for more details.
Container
You can customize the container you are using, which should have minimally Snakemake and your application software. We have prepared a container with Flux, Snakemake, and the various other plugins for you to get started. The Dockerfile is packaged here if you'd like to tweak it, e.g.,
rule hello_world:
output:
"...",
resources:
container="ghcr.io/rse-ops/mamba:snakemake",
kueue_operator="job"
shell:
"..."
Memory
The memory defined for Kubernetes is in a string format, and while we could ideally do a conversion for now we are lazy and ask you to define it directly:
rule a:
input: ...
output: ...
resources:
kueue_memory=200M1
shell:
"..."
Tasks
The Flux Operator can handle tasks for MPI, so you can set them as follows:
rule a:
input: ...
output: ...
resources:
kueue_tasks=1
shell:
"..."
For examples, check out the example directory.
Want to write a plugin?
If you are interested in writing your own plugin, instructions are provided via the snakemake-executor-plugin-interface.
License
HPCIC DevTools is distributed under the terms of the MIT license. All new contributions must be made under this license.
See LICENSE, COPYRIGHT, and NOTICE for details.
SPDX-License-Identifier: (MIT)
LLNL-CODE- 842614
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for snakemake_executor_plugin_kueue-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | cc5b565a155cae0c4dfbbd3a815f5dd0973bd3b8e3eb9a3179415df82bfe0fe0 |
|
MD5 | 62675a2792a7c3e4e82857e788fcedd3 |
|
BLAKE2b-256 | efaf98c803a8ff20e23e610219036df91ab23b93f1180d3257a0c255787a85db |
Hashes for snakemake_executor_plugin_kueue-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3fcf1bffbd8cb12e7654e52634c9020b5b547852ed6011133b4b6c23bae71d5d |
|
MD5 | ef3a27ef406c1c58eb116cb4d7e29766 |
|
BLAKE2b-256 | 5a2fa20b2bdcc989013ebac543f77961455fa375227c1c32e1c8223954343b49 |