Gateway Provisioners - a package containing kernel provisioners supporting the deployment of remote and resource-managed kernels.
Project description
Gateway Provisioners
Gateway Provisioners provides kernel provisioners that interact with kernels launched into resource-managed clusters or otherwise run remotely from the launching server. This functionality derives from Jupyter Enterprise Gateway's process proxy architecture. However, unlike process proxies, you do not need to use a gateway server to use these provisioners - although, in certain cases, it is recommended (for example when the launching server does not reside within the same network as the launched kernel).
Here is the current set of provisioners provided by this package, many of which have their requirements conditionally installed:
KubernetesProvisioner
- Kernels (residing in images) are launched as pods within a Kubernetes clusterpip install gateway_provisioners[k8s]
DockerSwarmProvisioner
- Kernels (residing in images) are launched as containers within a DockerSwarm clusterDockerProvisioner
- Kernels (residing in images) are launched as containers on the local serverpip install gateway_provisioners[docker]
YarnProvisioner
- Kernels are launched into a Hadoop YARN cluster (primarily Spark)pip install gateway_provisioners[yarn]
DistributedProvisioner
- Kernels are launched across a set of hosts using SSH, round-robinpip install gateway_provisioners
This package also includes command-line utilities that can be used to create kernel specifications or inject bootstrap files into docker images relative to the desired provisioner:
jupyter-k8s-spec
- for building kernel specifications relative to theKubernetesProvisioner
jupyter-docker-spec
- for building kernel specifications relative toDockerProvisioner
andDockerSwarmProvisioner
jupyter-yarn-spec
- for building kernel specifications relative to theYarnProvisioner
jupyter-ssh-spec
- for building kernel specifications relative to theDistributedProvisioner
jupyter-image-bootstrap
- for injecting bootstrap support when building kernel-based images
Note: The container-based provisioners (KubernetesProvisioner
, DockerSwarmProvisioner
, and DockerProvisioner
)
require that the hosting server also be running within the same environment/network. As a result, these provisioners
may be better suited for use by a Gateway Server (e.g., Jupyter Kernel Gateway)
so as to not require the Notebook/Lab server also be deployed in a container.
Installation
Detailed deployment instructions are located in the
Operators Guide
of the project docs. Here's a quick start using pip
:
# install from pypi
pip install --upgrade gateway-provisioners
# options for the command-line utilities can be viewed using '--help-all'
jupyter yarn-spec install --help-all
# run it with default options to install a Python-based kernelspec for Hadoop Yarn
jupyter yarn-spec install
Contributing
The Contribution page includes information about how to contribute to Gateway Provisioners. We encourage you to explore the other topics in our Contributors Guide like how to set up a development environment, or gaining an understanding of the system architecture, among other things.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file gateway_provisioners-0.4.0.tar.gz
.
File metadata
- Download URL: gateway_provisioners-0.4.0.tar.gz
- Upload date:
- Size: 264.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/5.0.0 CPython/3.11.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 91ce4f26dcf46ef00aa7dac3688b89f72dfd26759e86017e643a5d73b6bd2d79 |
|
MD5 | 31df353f12339f4494e3b033a8adb0bd |
|
BLAKE2b-256 | b62e16f593ebfee8b1705bc9013fca66810482b9dd74fea7598f7da7d777f7fb |
Provenance
File details
Details for the file gateway_provisioners-0.4.0-py3-none-any.whl
.
File metadata
- Download URL: gateway_provisioners-0.4.0-py3-none-any.whl
- Upload date:
- Size: 222.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/5.0.0 CPython/3.11.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | d86a7e8da6631889de6d6070d027ddfa0343a9f3cd446755736177ac92c4c377 |
|
MD5 | 4b49fdb11c7f7b6b2b0761e2cc0fb05f |
|
BLAKE2b-256 | 2fc0c8646ad7263dd6cfc772fa09c5b993087dc47790f57c0f137e57cd94dd1d |