Skip to main content

Setup and manage a toil and Apache Mesos cluster in EC2

Project description

The CGCloud plugin for Toil lets you setup a fully configured Toil/Mesos cluster in EC2 in just minutes, regardless of the number of nodes.

Prerequisites

The cgcloud-toil package requires that the cgcloud-core package and its prerequisites are present.

Installation

Read the entire section before pasting any commands and ensure that all prerequisites are installed. It is recommended to install this plugin into the virtualenv you created for CGCloud:

source ~/cgcloud/bin/activate
pip install cgcloud-toil

If you get DistributionNotFound: No distributions matching the version for cgcloud-toil, try running pip install --pre cgcloud-toil.

Be sure to configure cgcloud-core before proceeding.

Configuration

Modify your .profile or .bash_profile by adding the following line:

export CGCLOUD_PLUGINS="cgcloud.toil:$CGCLOUD_PLUGINS"

Login and out (or, on OS X, start a new Terminal tab/window).

Verify the installation by running:

cgcloud list-roles

The output should include the toil-box role.

Usage

Create a single t2.micro box to serve as the template for the cluster nodes:

cgcloud create -IT toil-box

The I option stops the box once it is fully set up and takes an image (AMI) of it. The T option terminates the box after that.

Substitute toil-latest-box for toil-box if you want to use the latest unstable release of Toil.

Now create a cluster by booting a leader and the workers from that AMI:

cgcloud create-cluster toil -s 2 -t m3.large

This will launch a leader and two workers using the m3.large instance type.

SSH into the leader:

cgcloud ssh toil-leader

… or the first worker:

cgcloud ssh -o 0 toil-worker

… or the second worker:

cgcloud ssh -o 1 toil-worker

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cgcloud-toil-1.4a1.dev312.tar.gz (6.7 kB view details)

Uploaded Source

Built Distribution

cgcloud_toil-1.4a1.dev312-py2.7.egg (15.5 kB view details)

Uploaded Source

File details

Details for the file cgcloud-toil-1.4a1.dev312.tar.gz.

File metadata

File hashes

Hashes for cgcloud-toil-1.4a1.dev312.tar.gz
Algorithm Hash digest
SHA256 08995c25477c8762a465717be647e1d908a38344684e1289d0284ac20c268ef5
MD5 070b46d4859f75e9f6ddae39064468ed
BLAKE2b-256 b260813c2b381fdc495c5b4e9ea41e41a2ad9ef50dd67e833b0074961a088e2e

See more details on using hashes here.

File details

Details for the file cgcloud_toil-1.4a1.dev312-py2.7.egg.

File metadata

File hashes

Hashes for cgcloud_toil-1.4a1.dev312-py2.7.egg
Algorithm Hash digest
SHA256 a74cfe6045fe19e51045360bdf21b7c94eec216293f2aa0a18637db057a836ca
MD5 adfb28f7f0589247889e7fd8e46bf777
BLAKE2b-256 319091c2a90cb3643ac32fe9fa58a319be55756bb5279c7951b9161ac418e1de

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page