Dask Cluster objects in Saturn Cloud
Project description
dask-saturn
Python library for interacting with Dask clusters in Saturn Cloud.
Dask-Saturn mimics the API of Dask-Kubernetes, but allows the user to interact with clusters created within Saturn Cloud.
Start cluster
In order to interact with a Dask cluster, the cluster must first be created in the Saturn User Interface. Then, from within a Jupyter notebook, you can start the cluster and adjust the number of workers.
from dask_saturn import SaturnCluster
cluster = SaturnCluster
cluster
Adjust number of workers
Once you have a cluster you can interact with it via the jupyter
widget, or using the scale
and adapt
methods.
For example, to manually scale up to 20 workers:
cluster.scale(20)
To create an adaptive cluster that controls its own scaling:
cluster.adapt(minimum=1, maximum=20)
Interact with client
To submit tasks to the cluster, you sometimes need access to the
Client
object. Instantiate this with the cluster as the only argument:
from distributed import Client
client = Client(cluster)
client
Close cluster
To terminate all resources associated with a cluster, use the
close
method:
cluster.close()
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for dask_saturn-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a96be524992573fbb70d20a61730187de3a0495533da593621877d6566073ea6 |
|
MD5 | 87d065eb39a93c63991f9cd4a6dc21c3 |
|
BLAKE2b-256 | d497f04815374658d41be833cfc5ad220ae76322f1d22bc1385dcd8283304c6f |