Skip to main content

KServe Python SDK

Project description

KServe Python SDK

Python SDK for KServe Server and Client.

Installation

KServe Python SDK can be installed by pip or poetry.

pip install

pip install kserve

To install Kserve with storage support

pip install kserve[storage]

Poetry

Install via Poetry.

make dev_install

To install Kserve with storage support

poetry install -E storage

or

poetry install --extras "storage"

KServe Python Server

KServe's python server libraries implement a standardized library that is extended by model serving frameworks such as Scikit Learn, XGBoost and PyTorch. It encapsulates data plane API definitions and storage retrieval for models.

It provides many functionalities, including among others:

  • Registering a model and starting the server
  • Prediction Handler
  • Pre/Post Processing Handler
  • Liveness Handler
  • Readiness Handlers

It supports the following storage providers:

  • Google Cloud Storage with a prefix: "gs://"
    • By default, it uses GOOGLE_APPLICATION_CREDENTIALS environment variable for user authentication.
    • If GOOGLE_APPLICATION_CREDENTIALS is not provided, anonymous client will be used to download the artifacts.
  • S3 Compatible Object Storage with a prefix "s3://"
    • By default, it uses S3_ENDPOINT, AWS_ACCESS_KEY_ID, and AWS_SECRET_ACCESS_KEY environment variables for user authentication.
  • Azure Blob Storage with the format: "https://{$STORAGE_ACCOUNT_NAME}.blob.core.windows.net/{$CONTAINER}/{$PATH}"
  • Local filesystem either without any prefix or with a prefix "file://". For example:
    • Absolute path: /absolute/path or file:///absolute/path
    • Relative path: relative/path or file://relative/path
    • For local filesystem, we recommended to use relative path without any prefix.
  • Persistent Volume Claim (PVC) with the format "pvc://{$pvcname}/[path]".
    • The pvcname is the name of the PVC that contains the model.
    • The [path] is the relative path to the model on the PVC.
    • For e.g. pvc://mypvcname/model/path/on/pvc
  • Generic URI, over either HTTP, prefixed with http:// or HTTPS, prefixed with https://. For example:
    • https://<some_url>.com/model.joblib
    • http://<some_url>.com/model.joblib

Metrics

For latency metrics, send a request to /metrics. Prometheus latency histograms are emitted for each of the steps (pre/postprocessing, explain, predict). Additionally, the latencies of each step are logged per request.

Metric Name Description Type
request_preprocess_seconds pre-processing request latency Histogram
request_explain_seconds explain request latency Histogram
request_predict_seconds prediction request latency Histogram
request_postprocess_seconds pre-processing request latency Histogram

KServe Client

Getting Started

KServe's python client interacts with KServe control plane APIs for executing operations on a remote KServe cluster, such as creating, patching and deleting of a InferenceService instance. See the Sample for Python SDK Client to get started.

Documentation for Client API

Please review KServe Client API docs.

Documentation For Models

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

kserve-0.11.0rc0.tar.gz (230.2 kB view details)

Uploaded Source

Built Distribution

kserve-0.11.0rc0-py3-none-any.whl (337.0 kB view details)

Uploaded Python 3

File details

Details for the file kserve-0.11.0rc0.tar.gz.

File metadata

  • Download URL: kserve-0.11.0rc0.tar.gz
  • Upload date:
  • Size: 230.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.4.2 CPython/3.9.16 Darwin/22.4.0

File hashes

Hashes for kserve-0.11.0rc0.tar.gz
Algorithm Hash digest
SHA256 1a79e0866574f4a50a4a4912d421940437a9c393e62112bacf96ebed19077d92
MD5 c71e6ea723b94c0b08073f57c66851fb
BLAKE2b-256 e247828e74b30e86b33b10dd95b27ee8e1d75038fb9f63a707dd6ab222252788

See more details on using hashes here.

File details

Details for the file kserve-0.11.0rc0-py3-none-any.whl.

File metadata

  • Download URL: kserve-0.11.0rc0-py3-none-any.whl
  • Upload date:
  • Size: 337.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.4.2 CPython/3.9.16 Darwin/22.4.0

File hashes

Hashes for kserve-0.11.0rc0-py3-none-any.whl
Algorithm Hash digest
SHA256 f478a97001cea11baf3f6a339a897e4e2a0fb9086aec6ff941ae4467202f485f
MD5 2c2850609c7c2818ba1851fb2ce89283
BLAKE2b-256 97b5e2db7ad0f476569a15df4fa8a2e44737c4c82e030c0ab2319b25a34f2dac

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page