Dask Cluster objects in Saturn Cloud
Project description
dask-saturn
Python library for interacting with Dask clusters in Saturn Cloud.
Dask-Saturn mimics the API of Dask-Kubernetes, but allows the user to interact with clusters created within Saturn Cloud.
Start cluster
In order to interact with a Dask cluster, the cluster must first be created in the Saturn User Interface. Then, from within a Jupyter notebook, you can start the cluster and adjust the number of workers.
from dask_saturn import SaturnCluster
cluster = SaturnCluster()
cluster
Adjust number of workers
Once you have a cluster you can interact with it via the jupyter
widget, or using the scale and adapt methods.
For example, to manually scale up to 20 workers:
cluster.scale(20)
To create an adaptive cluster that controls its own scaling:
cluster.adapt(minimum=1, maximum=20)
Interact with client
To submit tasks to the cluster, you sometimes need access to the
Client object. Instantiate this with the cluster as the only argument:
from distributed import Client
client = Client(cluster)
client
Close cluster
To terminate all resources associated with a cluster, use the
close method:
cluster.close()
Development
Create/update a dask-saturn conda environment:
make conda-update
Set environment variables to run dask-saturn with a local atlas server:
export BASE_URL=http://dev.localtest.me:8888/
export SATURN_TOKEN=<JUPYTER_SERVER_SATURN_TOKEN>
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for dask_saturn-0.0.3-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 0753db8f239e93da879fbf82a90bee74bb6b3d409a217b235acca0839e345f2a |
|
| MD5 | 2189a51221fe16cacd8d9412dad2caf1 |
|
| BLAKE2b-256 | ed336c3c316cbe574065eb3feed6168fb8f588551f0fc32518a4d08f4908bc2d |
