Skip to main content

Gateway Provisioners - a package containing kernel provisioners supporting the deployment of remote and resource-managed kernels.

Project description

Gateway Provisioners

Build Status Documentation Status

Gateway Provisioners provides kernel provisioners that interact with kernels launched into resource-managed clusters or otherwise run remotely from the launching server. This functionality derives from Jupyter Enterprise Gateway's process proxy architecture. However, unlike process proxies, you do not need to use a gateway server to use these provisioners - although, in certain cases, it is recommended (for example when the launching server does not reside within the same network as the launched kernel).

Here is the current set of provisioners provided by this package, many of which have their requirements conditionally installed:

  • KubernetesProvisioner - Kernels (residing in images) are launched as pods within a Kubernetes cluster
    • pip install gateway_provisioners[k8s]
  • DockerSwarmProvisioner - Kernels (residing in images) are launched as containers within a DockerSwarm cluster
  • DockerProvisioner - Kernels (residing in images) are launched as containers on the local server
    • pip install gateway_provisioners[docker]
  • YarnProvisioner - Kernels are launched into a Hadoop YARN cluster (primarily Spark)
    • pip install gateway_provisioners[yarn]
  • DistributedProvisioner - Kernels are launched across a set of hosts using SSH, round-robin
    • pip install gateway_provisioners

This package also includes command-line utilities that can be used to create kernel specifications or inject bootstrap files into docker images relative to the desired provisioner:

  • jupyter-k8s-spec - for building kernel specifications relative to the KubernetesProvisioner
  • jupyter-docker-spec - for building kernel specifications relative to DockerProvisioner and DockerSwarmProvisioner
  • jupyter-yarn-spec - for building kernel specifications relative to the YarnProvisioner
  • jupyter-ssh-spec - for building kernel specifications relative to the DistributedProvisioner
  • jupyter-image-bootstrap - for injecting bootstrap support when building kernel-based images

Note: The container-based provisioners (KubernetesProvisioner, DockerSwarmProvisioner, and DockerProvisioner) require that the hosting server also be running within the same environment/network. As a result, these provisioners may be better suited for use by a Gateway Server (e.g., Jupyter Kernel Gateway) so as to not require the Notebook/Lab server also be deployed in a container.


Installation

Detailed deployment instructions are located in the Operators Guide of the project docs. Here's a quick start using pip:

# install from pypi
pip install --upgrade gateway-provisioners

# options for the command-line utilities can be viewed using '--help-all'
jupyter yarn-spec install --help-all

# run it with default options to install a Python-based kernelspec for Hadoop Yarn
jupyter yarn-spec install

Contributing

The Contribution page includes information about how to contribute to Gateway Provisioners. We encourage you to explore the other topics in our Contributors Guide like how to set up a development environment, or gaining an understanding of the system architecture, among other things.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gateway_provisioners-0.4.0.tar.gz (264.9 kB view details)

Uploaded Source

Built Distribution

gateway_provisioners-0.4.0-py3-none-any.whl (222.9 kB view details)

Uploaded Python 3

File details

Details for the file gateway_provisioners-0.4.0.tar.gz.

File metadata

  • Download URL: gateway_provisioners-0.4.0.tar.gz
  • Upload date:
  • Size: 264.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.0.0 CPython/3.11.8

File hashes

Hashes for gateway_provisioners-0.4.0.tar.gz
Algorithm Hash digest
SHA256 91ce4f26dcf46ef00aa7dac3688b89f72dfd26759e86017e643a5d73b6bd2d79
MD5 31df353f12339f4494e3b033a8adb0bd
BLAKE2b-256 b62e16f593ebfee8b1705bc9013fca66810482b9dd74fea7598f7da7d777f7fb

See more details on using hashes here.

Provenance

File details

Details for the file gateway_provisioners-0.4.0-py3-none-any.whl.

File metadata

File hashes

Hashes for gateway_provisioners-0.4.0-py3-none-any.whl
Algorithm Hash digest
SHA256 d86a7e8da6631889de6d6070d027ddfa0343a9f3cd446755736177ac92c4c377
MD5 4b49fdb11c7f7b6b2b0761e2cc0fb05f
BLAKE2b-256 2fc0c8646ad7263dd6cfc772fa09c5b993087dc47790f57c0f137e57cd94dd1d

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page