Skip to main content

A minimum-lovable machine-learning pipeline, built on top of AWS SageMaker.

Project description

ML2P – or (ML)^2P – is the minimal lovable machine-learning pipeline and a friendlier interface to AWS SageMaker.

Design goals:

  • support the full machine learning lifecyle

  • support custom feature engineering

  • support building custom models in Python

  • provide reproducible training and deployment of models

  • support the use of customised base Docker images for training and deployment

Concretely it provides a command line interface and a Python library to assist with:

  • S3:
    • Managing training data

  • SageMaker:
    • Launching training jobs

    • Deploying trained models

    • Creating notebook instances

  • On your local machine or in a SageMaker notebook:
    • Downloading training datasets from S3

    • Training models

    • Loading trained models from SageMaker / S3

Installing

Install ML2P with:

$ pip install ml2p

Mailing list

If you have questions about ML2P, or would like to contribute or have suggestions for improvements, you are welcome to join the project mailing list at https://groups.google.com/g/ml2p and write us a letter there.

Overview

ML2P helps manage a machine learning project. You’ll define your project by writing a small YAML file named ml2p.yml:

project: "ml2p-tutorial"
s3folder: "s3://your-s3-bucket/"
models:
  bob: "models.RegressorModel"
defaults:
  image: "XXXXX.dkr.ecr.REGION.amazonaws.com/your-docker-image:X.Y.Z"
  role: "arn:aws:iam::XXXXX:role/your-role"
train:
  instance_type: "ml.m5.large"
deploy:
  instance_type: "ml.t2.medium"
  record_invokes: true

This specifies:

  • project: the name of your project

  • s3folder: the S3 bucket that will hold the models and data sets for your project

  • models: a list of model names and the Python classes that will be used to train the models and make predictions

  • defaults:

    • image: the docker image that your project will use for training and prediction

    • role: the AWS role your project will run under

  • train:

    • instance_type: the AWS instance type that will be used when training your model

  • deploy:

    • instance_type: the AWS instance type that will be used when deploying your model

    • record_invokes: whether to record prediction requests in S3

The name of your project functions as a prefix to the names of SageMaker training jobs, models and endpoints that ML2P creates (since these names are global within a SageMaker account).

ML2P also tags all of the AWS objects it creates with your project name.

Tutorial

See https://ml2p.readthedocs.io/en/latest/tutorial/ for a step-by-step tutorial.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ml2p-0.2.2.tar.gz (37.9 kB view details)

Uploaded Source

Built Distribution

ml2p-0.2.2-py2.py3-none-any.whl (39.9 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file ml2p-0.2.2.tar.gz.

File metadata

  • Download URL: ml2p-0.2.2.tar.gz
  • Upload date:
  • Size: 37.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.5

File hashes

Hashes for ml2p-0.2.2.tar.gz
Algorithm Hash digest
SHA256 6b8bb36dd8cf76cfa54b5904cbfda428610649f78eae783159c5dc44f2f32720
MD5 736d56ee766e1e39a4808c62a128f3f5
BLAKE2b-256 5669a62e4d79b6b7cea4addcbe5f03db82b087671bb58f5600978af70f55f66c

See more details on using hashes here.

File details

Details for the file ml2p-0.2.2-py2.py3-none-any.whl.

File metadata

  • Download URL: ml2p-0.2.2-py2.py3-none-any.whl
  • Upload date:
  • Size: 39.9 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.5

File hashes

Hashes for ml2p-0.2.2-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 94a55afeae3f6f1fd93a59ef6ca51cd7d7ae6fb67a3c4066a939b455f1e8c1c0
MD5 ab80ddf2700a84f1df5c9073461d7951
BLAKE2b-256 4eba8fbb7e173b9ace2bc59cc4b6ae64f150a060c56387f67003a5dfb292cae8

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page