Table of Contents

• Preface
• neuro
  • neuro admin
    • neuro admin get-clusters
    • neuro admin generate-cluster-config
    • neuro admin add-cluster
    • neuro admin get-cluster-users
    • neuro admin add-cluster-user
    • neuro admin remove-cluster-user
    • neuro admin set-user-quota
    • neuro admin add-user-quota
  • neuro job
    • neuro job run
    • neuro job submit
    • neuro job ls
    • neuro job status
    • neuro job exec
    • neuro job port-forward
    • neuro job logs
    • neuro job kill
    • neuro job top
    • neuro job save
    • neuro job browse
  • neuro project
    • neuro project init
  • neuro storage
    • neuro storage cp
    • neuro storage ls
    • neuro storage glob
    • neuro storage rm
    • neuro storage mkdir
    • neuro storage mv
    • neuro storage load
  • neuro image
    • neuro image ls
    • neuro image push
    • neuro image pull
    • neuro image tags
  • neuro config
    • neuro config login
    • neuro config login-with-token
    • neuro config login-headless
    • neuro config show
    • neuro config show-token
    • neuro config show-quota
    • neuro config get-clusters
    • neuro config switch-cluster
    • neuro config docker
    • neuro config logout
  • neuro completion
    • neuro completion generate
    • neuro completion patch
  • neuro acl
    • neuro acl grant
    • neuro acl revoke
    • neuro acl list
  • neuro help
  • neuro run
  • neuro submit
  • neuro ps
  • neuro status
  • neuro exec
  • neuro port-forward
  • neuro logs
  • neuro kill
  • neuro top
  • neuro save
  • neuro login
  • neuro logout
  • neuro cp
  • neuro ls
  • neuro rm
  • neuro mkdir
  • neuro mv
  • neuro images
  • neuro push
  • neuro pull
  • neuro share
• Api
• Contributing

Preface

Welcome to Neuromation API Python client. Package ship command line tool called neuro. With neuro you can:

• Execute and debug jobs
• Manipulate Data
• Make some fun

neuro

Usage:

neuro [OPTIONS] COMMAND [ARGS]...

Options:

Command Groups:

Commands:

neuro admin

Cluster administration commands.

Usage:

neuro admin [OPTIONS] COMMAND [ARGS]...

Options:

Commands:

neuro admin get-clusters

Print the list of available clusters.

Usage:

neuro admin get-clusters [OPTIONS]

Options:

neuro admin generate-cluster-config

Create a cluster configuration file.

Usage:

neuro admin generate-cluster-config [OPTIONS] [CONFIG]

Options:

neuro admin add-cluster

Create a new cluster and start its provisioning.

Usage:

neuro admin add-cluster [OPTIONS] CLUSTER_NAME CONFIG

Options:

neuro admin get-cluster-users

Print the list of all users in the cluster with their assigned role.

Usage:

neuro admin get-cluster-users [OPTIONS] [CLUSTER_NAME]

Options:

neuro admin add-cluster-user

Add user access to specified cluster.

The command supports one of 3 user roles: admin, manager or user.

Usage:

neuro admin add-cluster-user [OPTIONS] CLUSTER_NAME USER_NAME [ROLE]

Options:

neuro admin remove-cluster-user

Remove user access from the cluster.

Usage:

neuro admin remove-cluster-user [OPTIONS] CLUSTER_NAME USER_NAME

Options:

neuro admin set-user-quota

Set user quota to given values

Usage:

neuro admin set-user-quota [OPTIONS] CLUSTER_NAME USER_NAME

Options:

neuro admin add-user-quota

Add given values to user quota

Usage:

neuro admin add-user-quota [OPTIONS] CLUSTER_NAME USER_NAME

Options:

neuro job

Job operations.

Usage:

neuro job [OPTIONS] COMMAND [ARGS]...

Options:

Commands:

neuro job run

Run a job with predefined resources configuration.

IMAGE container image name.

CMD list will be passed as commands to model container.

Usage:

neuro job run [OPTIONS] IMAGE [CMD]...

Examples:

# Starts a container pytorch:latest on a machine with smaller GPU resources
# (see exact values in `neuro config show`) and with two volumes mounted:
#   storage://<home-directory>   --> /var/storage/home (in read-write mode),
#   storage://neuromation/public --> /var/storage/neuromation (in read-only mode).
neuro run --preset=gpu-small --volume=HOME pytorch:latest

# Starts a container using the custom image my-ubuntu:latest stored in neuromation
# registry, run /script.sh and pass arg1 and arg2 as its arguments:
neuro run -s cpu-small image://~/my-ubuntu:latest --entrypoint=/script.sh arg1 arg2

Options:

neuro job submit

Submit an image to run on the cluster.

IMAGE container image name.

CMD list will be passed as commands to model container.

Usage:

neuro job submit [OPTIONS] IMAGE [CMD]...

Examples:

# Starts a container pytorch:latest with two paths mounted. Directory /q1/
# is mounted in read only mode to /qm directory within container.
# Directory /mod mounted to /mod directory in read-write mode.
neuro submit --volume storage:/q1:/qm:ro --volume storage:/mod:/mod:rw pytorch:latest

# Starts a container using the custom image my-ubuntu:latest stored in neuromation
# registry, run /script.sh and pass arg1 arg2 arg3 as its arguments:
neuro submit image://~/my-ubuntu:latest --entrypoint=/script.sh arg1 arg2 arg3

Options:

neuro job ls

List all jobs.

Usage:

neuro job ls [OPTIONS]

Examples:

neuro ps -a
neuro ps -a --owner=user-1 --owner=user-2
neuro ps --name my-experiments-v1 -s failed -s succeeded
neuro ps --description=my favourite job
neuro ps -s failed -s succeeded -q

Options:

neuro job status

Display status of a job.

Usage:

neuro job status [OPTIONS] JOB

Options:

neuro job exec

Execute command in a running job.

Usage:

neuro job exec [OPTIONS] JOB CMD...

Examples:

# Provides a shell to the container:
neuro exec my-job /bin/bash

# Executes a single command in the container and returns the control:
neuro exec --no-tty my-job ls -l

Options:

neuro job port-forward

Forward port(s) of a running job to local port(s).

Usage:

neuro job port-forward [OPTIONS] JOB LOCAL_REMOTE_PORT...

Examples:

# Forward local port 2080 to port 80 of job's container.
# You can use http://localhost:2080 in browser to access job's served http
neuro job port-forward my-fastai-job 2080:80

# Forward local port 2222 to job's port 22
# Then copy all data from container's folder '/data' to current folder
# (please run second command in other terminal)
neuro job port-forward my-job-with-ssh-server 2222:22
rsync -avxzhe ssh -p 2222 root@localhost:/data .

# Forward few ports at once
neuro job port-forward my-job- 2080:80 2222:22 2000:100

Options:

neuro job logs

Print the logs for a container.

Usage:

neuro job logs [OPTIONS] JOB

Options:

neuro job kill

Kill job(s).

Usage:

neuro job kill [OPTIONS] JOBS...

Options:

neuro job top

Display GPU/CPU/Memory usage.

Usage:

neuro job top [OPTIONS] JOB

Options:

neuro job save

Save job's state to an image.

Usage:

neuro job save [OPTIONS] JOB IMAGE

Examples:

neuro job save job-id image:ubuntu-patched
neuro job save my-favourite-job image://~/ubuntu-patched:v1
neuro job save my-favourite-job image://bob/ubuntu-patched

Options:

neuro job browse

Opens a job's URL in a web browser.

Usage:

neuro job browse [OPTIONS] JOB

Options:

neuro project

Project operations.

Usage:

neuro project [OPTIONS] COMMAND [ARGS]...

Options:

Commands:

neuro project init

Initialize an empty project.

Usage:

neuro project init [OPTIONS] [SLUG]

Examples:

# Initializes a scaffolding for the new project with the recommended project
# structure (see http://github.com/neuromation/cookiecutter-neuro-project)
neuro project init

# Initializes a scaffolding for the new project with the recommended project
# structure and sets default project folder name to "example"
neuro project init my-project-id

Options:

neuro storage

Storage operations.

Usage:

neuro storage [OPTIONS] COMMAND [ARGS]...

Options:

Commands:

neuro storage cp

Copy files and directories.

Either SOURCES or DESTINATION should have storage:// scheme. If scheme is
omitted, file:// scheme is assumed.

Use /dev/stdin and /dev/stdout file names to copy a file from terminal and
print the content of file on the storage to console.

Usage:

neuro storage cp [OPTIONS] [SOURCES]... [DESTINATION]

Examples:

# copy local files into remote storage root
neuro cp foo.txt bar/baz.dat storage:
neuro cp foo.txt bar/baz.dat -t storage:

# copy local directory `foo` into existing remote directory `bar`
neuro cp -r foo -t storage:bar

# copy the content of local directory `foo` into existing remote
# directory `bar`
neuro cp -r -T storage:foo storage:bar

# download remote file `foo.txt` into local file `/tmp/foo.txt` with
# explicit file:// scheme set
neuro cp storage:foo.txt file:///tmp/foo.txt
neuro cp -T storage:foo.txt file:///tmp/foo.txt
neuro cp storage:foo.txt file:///tmp
neuro cp storage:foo.txt -t file:///tmp

# download other user's remote file into the current directory
neuro cp storage://{username}/foo.txt .

# download only files with extension `.out` into the current directory
neuro cp storage:results/*.out .

Options:

neuro storage ls

List directory contents.

By default PATH is equal user's home dir (storage:)

Usage:

neuro storage ls [OPTIONS] [PATHS]...

Options:

neuro storage glob

List resources that match PATTERNS.

Usage:

neuro storage glob [OPTIONS] [PATTERNS]...

Options:

neuro storage rm

Remove files or directories.

Usage:

neuro storage rm [OPTIONS] PATHS...

Examples:

neuro rm storage:foo/bar
neuro rm storage://{username}/foo/bar
neuro rm --recursive storage://{username}/foo/
neuro rm storage:foo/**/*.tmp

Options:

neuro storage mkdir

Make directories.

Usage:

neuro storage mkdir [OPTIONS] PATHS...

Options:

neuro storage mv

Move or rename files and directories.

SOURCE must contain path to the file or directory existing on the storage,
and DESTINATION must contain the full path to the target file or directory.

Usage:

neuro storage mv [OPTIONS] [SOURCES]... [DESTINATION]

Examples:

# move and rename remote file
neuro mv storage:foo.txt storage:bar/baz.dat
neuro mv -T storage:foo.txt storage:bar/baz.dat

# move remote files into existing remote directory
neuro mv storage:foo.txt storage:bar/baz.dat storage:dst
neuro mv storage:foo.txt storage:bar/baz.dat -t storage:dst

# move the content of remote directory into other existing
# remote directory
neuro mv -T storage:foo storage:bar

# move remote file into other user's directory
neuro mv storage:foo.txt storage://{username}/bar.dat

# move remote file from other user's directory
neuro mv storage://{username}/foo.txt storage:bar.dat

Options:

neuro storage load

Copy files and directories using MinIO (EXPERIMENTAL).

Same as "cp", but uses MinIO and the Amazon S3 protocol.
(DEPRECATED)

Usage:

neuro storage load [OPTIONS] [SOURCES]... [DESTINATION]

Options:

neuro image

Container image operations.

Usage:

neuro image [OPTIONS] COMMAND [ARGS]...

Options:

Commands:

neuro image ls

List images.

Usage:

neuro image ls [OPTIONS]

Options:

neuro image push

Push an image to platform registry.

Remote image must be URL with image:// scheme. Image names can contain tag.
If tags not specified 'latest' will be used as value.

Usage:

neuro image push [OPTIONS] LOCAL_IMAGE [REMOTE_IMAGE]

Examples:

neuro push myimage
neuro push alpine:latest image:my-alpine:production
neuro push alpine image://myfriend/alpine:shared

Options:

neuro image pull

Pull an image from platform registry.

Remote image name must be URL with image:// scheme. Image names can contain
tag.

Usage:

neuro image pull [OPTIONS] REMOTE_IMAGE [LOCAL_IMAGE]

Examples:

neuro pull image:myimage
neuro pull image://myfriend/alpine:shared
neuro pull image://username/my-alpine:production alpine:from-registry

Options:

neuro image tags

List tags for image in platform registry.

Image name must be URL with image:// scheme.

Usage:

neuro image tags [OPTIONS] IMAGE

Examples:

neuro image tags image://myfriend/alpine
neuro image tags image:myimage

Options:

neuro config

Client configuration.

Usage:

neuro config [OPTIONS] COMMAND [ARGS]...

Options:

Commands:

neuro config login

Log into Neuro Platform.

URL is a platform entrypoint URL.

Usage:

neuro config login [OPTIONS] [URL]

Options:

neuro config login-with-token

Log into Neuro Platform with token.

TOKEN is authentication token provided by administration team. URL is a
platform entrypoint URL.

Usage:

neuro config login-with-token [OPTIONS] TOKEN [URL]

Options:

neuro config login-headless

Log into Neuro Platform from non-GUI server environment.

URL is a platform entrypoint URL.

The command works similar to "neuro login" but instead of opening a browser
for performing OAuth registration prints an URL that should be open on guest
host.

Then user inputs a code displayed in a browser after successful login back
in neuro command to finish the login process.

Usage:

neuro config login-headless [OPTIONS] [URL]

Options:

neuro config show

Print current settings.

Usage:

neuro config show [OPTIONS]

Options:

neuro config show-token

Print current authorization token.

Usage:

neuro config show-token [OPTIONS]

Options:

neuro config show-quota

Print quota and remaining computation time for active cluster.

Usage:

neuro config show-quota [OPTIONS] [USER]

Options:

neuro config get-clusters

Fetch and display the list of available clusters.

Usage:

neuro config get-clusters [OPTIONS]

Options:

neuro config switch-cluster

Switch the active cluster.

CLUSTER_NAME is the cluster name to select. The interactive prompt is used
if the name is omitted (default).

Usage:

neuro config switch-cluster [OPTIONS] [CLUSTER_NAME]

Options:

neuro config docker

Configure docker client to fit the Neuro Platform.

Usage:

neuro config docker [OPTIONS]

Options:

neuro config logout

Log out.

Usage:

neuro config logout [OPTIONS]

Options:

neuro completion

Output shell completion code.

Usage:

neuro completion [OPTIONS] COMMAND [ARGS]...

Options:

Commands:

neuro completion generate

Provide an instruction for shell completion generation.

Usage:

neuro completion generate [OPTIONS] [bash|zsh]

Options:

neuro completion patch

Automatically patch shell configuration profile to enable completion

Usage:

neuro completion patch [OPTIONS] [bash|zsh]

Options:

neuro acl

Access Control List management.

Usage:

neuro acl [OPTIONS] COMMAND [ARGS]...

Options:

Commands:

neuro acl grant

Shares resource with another user.

URI shared resource.

USER username to share resource with.

PERMISSION sharing access right: read, write, or manage.

Usage:

neuro acl grant [OPTIONS] URI USER [read|write|manage]

Examples:

neuro acl grant storage:///sample_data/ alice manage
neuro acl grant image:resnet50 bob read
neuro acl grant job:///my_job_id alice write

Options:

neuro acl revoke

Revoke user access from another user.

URI previously shared resource to revoke.

USER to revoke URI resource from.

Usage:

neuro acl revoke [OPTIONS] URI USER

Examples:

neuro acl revoke storage:///sample_data/ alice
neuro acl revoke image:resnet50 bob
neuro acl revoke job:///my_job_id alice

Options:

neuro acl list

List shared resources.

The command displays a list of resources shared BY current user (default).

To display a list of resources shared WITH current user apply --shared
option.

Usage:

neuro acl list [OPTIONS]

Examples:

neuro acl list
neuro acl list --scheme storage
neuro acl list --shared
neuro acl list --shared --scheme image

Options:

neuro help

Get help on a command.

Usage:

neuro help [OPTIONS] [COMMAND]...

Options:

neuro run

Run a job with predefined resources configuration.

IMAGE container image name.

CMD list will be passed as commands to model container.

Usage:

neuro run [OPTIONS] IMAGE [CMD]...

Examples:

# Starts a container pytorch:latest on a machine with smaller GPU resources
# (see exact values in `neuro config show`) and with two volumes mounted:
#   storage://<home-directory>   --> /var/storage/home (in read-write mode),
#   storage://neuromation/public --> /var/storage/neuromation (in read-only mode).
neuro run --preset=gpu-small --volume=HOME pytorch:latest

# Starts a container using the custom image my-ubuntu:latest stored in neuromation
# registry, run /script.sh and pass arg1 and arg2 as its arguments:
neuro run -s cpu-small image://~/my-ubuntu:latest --entrypoint=/script.sh arg1 arg2

Options:

neuro submit

Submit an image to run on the cluster.

IMAGE container image name.

CMD list will be passed as commands to model container.

Usage:

neuro submit [OPTIONS] IMAGE [CMD]...

Examples:

# Starts a container pytorch:latest with two paths mounted. Directory /q1/
# is mounted in read only mode to /qm directory within container.
# Directory /mod mounted to /mod directory in read-write mode.
neuro submit --volume storage:/q1:/qm:ro --volume storage:/mod:/mod:rw pytorch:latest

# Starts a container using the custom image my-ubuntu:latest stored in neuromation
# registry, run /script.sh and pass arg1 arg2 arg3 as its arguments:
neuro submit image://~/my-ubuntu:latest --entrypoint=/script.sh arg1 arg2 arg3

Options:

neuro ps

List all jobs.

Usage:

neuro ps [OPTIONS]

Examples:

neuro ps -a
neuro ps -a --owner=user-1 --owner=user-2
neuro ps --name my-experiments-v1 -s failed -s succeeded
neuro ps --description=my favourite job
neuro ps -s failed -s succeeded -q

Options:

neuro status

Display status of a job.

Usage:

neuro status [OPTIONS] JOB

Options:

neuro exec

Execute command in a running job.

Usage:

neuro exec [OPTIONS] JOB CMD...

Examples:

# Provides a shell to the container:
neuro exec my-job /bin/bash

# Executes a single command in the container and returns the control:
neuro exec --no-tty my-job ls -l

Options:

neuro port-forward

Forward port(s) of a running job to local port(s).

Usage:

neuro port-forward [OPTIONS] JOB LOCAL_REMOTE_PORT...

Examples:

# Forward local port 2080 to port 80 of job's container.
# You can use http://localhost:2080 in browser to access job's served http
neuro job port-forward my-fastai-job 2080:80

# Forward local port 2222 to job's port 22
# Then copy all data from container's folder '/data' to current folder
# (please run second command in other terminal)
neuro job port-forward my-job-with-ssh-server 2222:22
rsync -avxzhe ssh -p 2222 root@localhost:/data .

# Forward few ports at once
neuro job port-forward my-job- 2080:80 2222:22 2000:100

Options:

neuro logs

Print the logs for a container.

Usage:

neuro logs [OPTIONS] JOB

Options:

neuro kill

Kill job(s).

Usage:

neuro kill [OPTIONS] JOBS...

Options:

neuro top

Display GPU/CPU/Memory usage.

Usage:

neuro top [OPTIONS] JOB

Options:

neuro save

Save job's state to an image.

Usage:

neuro save [OPTIONS] JOB IMAGE

Examples:

neuro job save job-id image:ubuntu-patched
neuro job save my-favourite-job image://~/ubuntu-patched:v1
neuro job save my-favourite-job image://bob/ubuntu-patched

Options:

neuro login

Log into Neuro Platform.

URL is a platform entrypoint URL.

Usage:

neuro login [OPTIONS] [URL]

Options:

neuro logout

Log out.

Usage:

neuro logout [OPTIONS]

Options:

neuro cp

Copy files and directories.

Either SOURCES or DESTINATION should have storage:// scheme. If scheme is
omitted, file:// scheme is assumed.

Use /dev/stdin and /dev/stdout file names to copy a file from terminal and
print the content of file on the storage to console.

Usage:

neuro cp [OPTIONS] [SOURCES]... [DESTINATION]

Examples:

# copy local files into remote storage root
neuro cp foo.txt bar/baz.dat storage:
neuro cp foo.txt bar/baz.dat -t storage:

# copy local directory `foo` into existing remote directory `bar`
neuro cp -r foo -t storage:bar

# copy the content of local directory `foo` into existing remote
# directory `bar`
neuro cp -r -T storage:foo storage:bar

# download remote file `foo.txt` into local file `/tmp/foo.txt` with
# explicit file:// scheme set
neuro cp storage:foo.txt file:///tmp/foo.txt
neuro cp -T storage:foo.txt file:///tmp/foo.txt
neuro cp storage:foo.txt file:///tmp
neuro cp storage:foo.txt -t file:///tmp

# download other user's remote file into the current directory
neuro cp storage://{username}/foo.txt .

# download only files with extension `.out` into the current directory
neuro cp storage:results/*.out .

Options:

neuro ls

List directory contents.

By default PATH is equal user's home dir (storage:)

Usage:

neuro ls [OPTIONS] [PATHS]...

Options:

neuro rm

Remove files or directories.

Usage:

neuro rm [OPTIONS] PATHS...

Examples:

neuro rm storage:foo/bar
neuro rm storage://{username}/foo/bar
neuro rm --recursive storage://{username}/foo/
neuro rm storage:foo/**/*.tmp

Options:

neuro mkdir

Make directories.

Usage:

neuro mkdir [OPTIONS] PATHS...

Options:

neuro mv

Move or rename files and directories.

SOURCE must contain path to the file or directory existing on the storage,
and DESTINATION must contain the full path to the target file or directory.

Usage:

neuro mv [OPTIONS] [SOURCES]... [DESTINATION]

Examples:

# move and rename remote file
neuro mv storage:foo.txt storage:bar/baz.dat
neuro mv -T storage:foo.txt storage:bar/baz.dat

# move remote files into existing remote directory
neuro mv storage:foo.txt storage:bar/baz.dat storage:dst
neuro mv storage:foo.txt storage:bar/baz.dat -t storage:dst

# move the content of remote directory into other existing
# remote directory
neuro mv -T storage:foo storage:bar

# move remote file into other user's directory
neuro mv storage:foo.txt storage://{username}/bar.dat

# move remote file from other user's directory
neuro mv storage://{username}/foo.txt storage:bar.dat

Options:

neuro images

List images.

Usage:

neuro images [OPTIONS]

Options:

neuro push

Push an image to platform registry.

Remote image must be URL with image:// scheme. Image names can contain tag.
If tags not specified 'latest' will be used as value.

Usage:

neuro push [OPTIONS] LOCAL_IMAGE [REMOTE_IMAGE]

Examples:

neuro push myimage
neuro push alpine:latest image:my-alpine:production
neuro push alpine image://myfriend/alpine:shared

Options:

neuro pull

Pull an image from platform registry.

Remote image name must be URL with image:// scheme. Image names can contain
tag.

Usage:

neuro pull [OPTIONS] REMOTE_IMAGE [LOCAL_IMAGE]

Examples:

neuro pull image:myimage
neuro pull image://myfriend/alpine:shared
neuro pull image://username/my-alpine:production alpine:from-registry

Options:

neuro share

Shares resource with another user.

URI shared resource.

USER username to share resource with.

PERMISSION sharing access right: read, write, or manage.

Usage:

neuro share [OPTIONS] URI USER [read|write|manage]

Examples:

neuro acl grant storage:///sample_data/ alice manage
neuro acl grant image:resnet50 bob read
neuro acl grant job:///my_job_id alice write

Options:

Api

TODO

Contributing

git clone https://github.com/neuromation/platform-api-clients.git
cd platform-api-clients/python

For OSX users install coreutils to properly interpret shell commands:

brew install coreutils

Before you begin, it is recommended to have clean virtual environment installed:

python -m venv .env
source .env/bin/activate

Development flow:

• Install dependencies: make init
• Run tests: make test
• Lint: make lint
• Publish to pypi: make publish