A tool to manage servers through a central configuration. Plugins allow provisioning, configuration and other management tasks.
Project description
Overview
Ploy is a commandline-tool to provision, manage and control server instances. There are plugins for EC2 (ploy_ec2), FreeBSD Jails (ploy_ezjail) and more. You can create, delete, monitor and ssh into instances while ploy handles the details like ssh fingerprint checking. Additional plugins provide advanced functionality like integrating Fabric (ploy_fabric) and Ansible (ploy_ansible).
You can find the detailed documentation at http://ploy.readthedocs.org/en/latest/
Installation
ploy is best installed with easy_install, pip or with zc.recipe.egg in a buildout. It installs two scripts, ploy and ploy-ssh.
Configuration
All information about server instances is located in ploy.conf, which by default is looked up in etc/ploy.conf.
Plugins
Support for backends and further functionality is implemented by plugins. One plugin is included with ploy.
- plain
For regular servers accessible via ssh.
You can see which plugins are available in your current installation with ploy -v.
Plain
With plain instances you can put infos about servers into the configuration to benefit from some ploy features like ssh fingerprint checking and plugins like the Fabric integration.
Options
- host or ip
(required) The host name or address for the server.
- user
The default user for ssh connections. If it’s set to * then the current local user name is used.
- port
The ssh port number.
- ssh-host-keys or ssh-fingerprints
(required) The public ssh host keys (or fingerprints) of the server.
- ssh-host-keys
It’s best to use the public ssh host keys if available. You can either specify the path, or put in the contents directly (ssh-rsa AAAA...). Multiple host keys can be specified one per line, or separated by commas.
- ssh-fingerprints
If set to ask then manual interactive verification is enabled. If set to ignore then no verification is performed at all! Multiple fingerprints can be specified one per line, or separated by commas. The format of fingerprints is either 16 bytes as hex numbers separated by colons, the same 16 bytes prefixed by MD5:, or the hash type, followed by a colon and the base 64 encoded hash digest.
- password-fallback
If this boolean is true, then using a password as fallback is enabled if the ssh key doesn’t work. This is off by default. You may be asked more than once for the password. The first time is by paramiko which always happens, but is remembered. The other times is by the ssh command line tool if it’s invoked.
- password
Never use this directly! If password-fallback is enabled this password is used. This is mainly meant for Fabric scripts which have other ways to get the password. On use case is bootstrapping FreeBSD from an mfsBSD distribution where the password is fixed.
- proxycommand
The command to use in the ProxyCommand option for ssh when using the ploy-ssh command. There are some variables which can be used:
- path
The directory of the ploy.conf file. Useful if you want to use the ploy-ssh command itself for the proxy.
- known_hosts
The absolute path to the known_hosts file managed by ploy.
- instances
The variables of other instances. For example: instances.foo.ip
In addition to these the variables of the instance itself are available.
A full example for a proxycommand:
proxycommand = {path}/../bin/ploy-ssh vm-master -W {ip}:22- ssh-key-filename
Location of private ssh key to use.
- ssh-extra-args
A list of settings separated by newlines passed on to ssh.
Example:
ssh-extra-args = ForwardAgent yes
SSH integration
ploy provides an additional tool ploy-ssh to easily perform SSH based operations against named instances. Particularly, it encapsulates the entire SSH fingerprint mechanism. For example EC2 instances are often short-lived and normally trigger warnings, especially, if you are using elastic IPs.
Note:: it does so not by simply turning off these checks, but by transparently updating its own fingerprint list using mechanisms provided by the backend plugins.
The easiest scenario is simply to create an SSH session with an instance. You can either use the ssh subcommand of the ploy tool like so:
ploy ssh INSTANCENAME
Alternatively you can use the ploy-ssh command directly, like so:
ploy-ssh INSTANCENAME
The latter has been provided to support scp and rsync. Here are some examples, you get the idea:
scp -S `pwd`/bin/ploy-ssh some.file demo-server:/some/path/ rsync -e "bin/ploy-ssh" some/path fschulze@demo-server:/some/path
Instance names
Instances have an id which is the part after the colon in the configuration. They also have a unique id which has the form [masterid]-[instanceid]. The [masterid] depends on the plugin. For plain instances it is plain. The [instanceid] is the id of the instance. So, if you have the following config:
[plain-instance:foo] ...
Then the unique id of the instance is plain-foo.
Macro expansion
In the ploy.conf you can use macro expansion for cleaner configuration files. That looks like this:
[ec2-instance:demo-server2] <= demo-server securitygroups = demo-server2 [ec2-securitygroup:demo-server2] <= demo-server
All the options from the specified macro are copied with some exceptions depending on the backend plugin.
If you want to copy data from some other kind of options, you can add a colon in the macro name. This is useful if you want to have a base for instances like this:
[macro:base-instance] keypair = default region = eu-west-1 placement = eu-west-1a [ec2-instance:server] <= macro:base-instance ...
Massaging of config values
Plugins and ploy massage certain string values from the config to convert them to other types and do formatting or expansion.
You can use that yourself, which is useful for the Fabric integration and other things.
Here is a simple example:
[section] massagers = intvalue=ploy.config.IntegerMassager boolvalue=ploy.config.BooleanMassager intvalue = 1 boolvalue = yes
If you now access those values from for example a fabric task, you get the correct type instead of strings.
The above syntax registers the massagers only for that section. You can register massagers for other sections or even section groups with this syntax:
massagers = [option]=[sectiongroup]:import.path.to.massager [option]=[sectiongroup]:[section]:import.path.to.massager
The parts have the following meaning:
- [option]
This is the name of the option which should be massaged
- [sectiongroup]
The name of the section group. That’s the part before the optional colon in a section. To match sections without a colon, use global. To match every section, use *.
- [section]
The name of the section to which this massager is applied. If empty, the current section is used.
Buildout specifics
With zc.recipe.egg you can set a custom configfile location like this:
[ploy]
recipe = zc.recipe.egg
eggs = ploy
arguments = configpath="${buildout:directory}/etc/", configname="servers.cfg"
Changelog
2.0.1 - 2023-06-19
Fix ssh timeout handling. [witsch, fschulze]
Fix getargspec deprecation warning with Python 3.x. [kappeck, fschulze]
2.0.0 - 2022-08-17
Dropped support for Python 3.6. [fschulze]
Added support for Python 3.10. [fschulze]
2.0.0b5 - 2021-07-17
Make a proper copy of the configuration for instances with multiple masters. [fschulze]
Use format_exc for error reporting in ssh command for more details. [fschulze]
2.0.0b4 - 2020-05-13
Fix multiple master association for [instance:...] sections. [fschulze]
2.0.0b3 - 2019-06-09
Some Python 3.x encoding related fixes. [fschulze]
2.0.0b2 - 2019-06-03
Add 5 seconds default timeout to ssh connections. [fschulze]
Dropped support for Python 3.4. [fschulze]
2.0.0b1 - 2018-02-06
Support YAML configuration files. [fschulze]
Dropped support ssh lib, only paramiko is supported. [fschulze]
Dropped support for Python 2.6 and 3.3. [fschulze]
1.5.2 - 2018-08-20
Stop testing on Python 3.3. [fschulze]
Fix multiple masters for instance. [fschulze]
1.5.1 - 2017-12-17
Fix getting fingerprints automatically when ssh-host-keys is used. [fschulze]
1.5.0 - 2017-10-03
New ssh-host-keys option which allows to set host keys directly to force paramiko to use a specific key type to avoid fingerprint mismatches. [fschulze]
1.4.0 - 2017-10-02
Allow instance implementations to return multiple fingerprints via get_fingerprints method. [fschulze]
Support other key types than just rsa, because paramiko > 2 defaults to ed25519. [fschulze]
Output more information about used keys on fingerprint mismatch. [fschulze]
1.3.1 - 2016-06-02
Don’t get ssh-fingerprints from master if not set in instance config. This fixes automatic fingerprint fetching for EC2, ezjail etc. [fschulze]
1.3.0 - 2016-05-21
Add option ssh-fingerprints which allows to specify multiple fingerprints. [fschulze]
Support new output of ssh-keygen which includes the hash type and defaults to SHA256. [fschulze]
1.2.1 - 2015-08-27
Allow to specify multiple masters per instance. [fschulze]
1.2.0 - 2015-03-05
Add Executor helper to handle local and remote command execution. It’s also handling ssh agent forwarding enabled by either the users ssh config or the ssh-extra-args option. [fschulze]
1.1.0 - 2015-02-28
Add ssh-extra-args option. [fschulze]
Add annotate command to print the configuration with the source of each setting. [fschulze]
Allow custom shebang in gzipped startup scripts. [fschulze]
1.0.3 - 2015-01-22
Drop bad entries from our known_hosts file to prevent failures in paramiko. [fschulze]
Set StrictHostKeyChecking=yes for all ssh connections to prevent interactive asking. [fschulze]
1.0.2 - 2014-10-04
Ask before terminating an instance. [fschulze]
Fix config setting propagation in some cases of proxied instances. [fschulze]
Close all connections before exiting. This prevents hangs caused by open proxy command threads. [fschulze]
Add option to log debug output. [fschulze]
Add helpers to setup proxycommand in plugins. [fschulze]
1.0.1 - 2014-08-13
Fix error output for plain instances on ssh connection failures. [fschulze]
1.0.0 - 2014-07-19
Fix removal of bad host keys when using non standard ssh port. [fschulze]
Renamed plain-master to plain, so the uids of instances are nicer. [fschulze]
1.0rc15 - 2014-07-16
Only remove bad host key from known_hosts instead of clearing it completely. [fschulze]
Removed support for proxyhost option. It caused hangs and failures on missing or invalid ssh fingerprints. [fschulze]
Allow empty startup_script option to mean use no startup script. [fschulze]
1.0rc14 - 2014-07-15
Allow fingerprint to be set to a public host key file. [fschulze]
1.0rc13 - 2014-07-08
Better error message for instances missing because the plugin isn’t installed. [fschulze]
Fix tests when ploy itself isn’t installed. [fschulze]
1.0rc12 - 2014-07-08
Use plain conftest.py instead of pytest plugin. [fschulze]
1.0rc11 - 2014-07-05
Fix uid method for master instances. [fschulze]
1.0rc10 - 2014-07-04
Print plugin versions with -v and --versions. [fschulze]
Python 3 compatibility. [fschulze]
1.0rc9 - 2014-06-29
Let plugins add type of lists to show with the list command. [fschulze]
Use server and instance consistently. [fschulze]
Always make instances accessible by their full name in the form of “[master_id]-[instance_id]”. Only if there is no conflict, the short version with just “[instance_id]” is also available for convenience. [fschulze]
Add instance id validator which limits to letters, numbers, dashes and underscores. [fschulze]
Renamed from mr.awsome to ploy. [fschulze]
1.0rc8 - 2014-06-16
Give a bit more info on ssh connection failures. [fschulze]
1.0rc7 - 2014-06-11
Expose some test fixtures for reuse in plugins. [fschulze]
Add before_terminate and after_start hooks and make it simple for plugins to add their own hooks. [fschulze]
1.0rc6 - 2014-06-10
Add get_path method to ConfigSection class. [fschulze]
1.0rc5 - 2014-06-09
Provide helper method ssh_args_from_info on BaseInstance to get the arguments for running the ssh executable from the info provided by init_ssh_key. [fschulze]
Allow overwriting the command name in help messages for bsdploy. [fschulze]
Make debug command usable for instances that don’t have a startup script. [fschulze]
Instances can provide a get_port method to return a default port. [fschulze]
Catch socket errors in init_ssh_key of plain instances to print additional info for debugging. [fschulze]
Delay setting of config file path to expose too early use of config in plugins. Refs #29 [fschulze]
1.0rc4 - 2014-05-21
Fix massagers for [instance:...] sections. [fschulze]
Copy massagers in ConfigSection.copy, so overrides in startup script work correctly. [fschulze]
1.0rc3 - 2014-05-15
Fetch fingerprints only when necessary. This speeds up connections when the fingerprint in known_hosts is still valid. [fschulze]
1.0rc2 - 2014-05-14
Moved setuptools-git from setup.py to .travis.yml, it’s only needed for releases and testing. [fschulze]
More tests. [fschulze]
1.0rc1 - 2014-03-23
Test, enhance and document adding massagers via config. [fschulze]
Moved ec2 and fabric integration into separate plugins. [fschulze]
You can now have instances with the same name if the belong to different masters, they will then get the name of the master as a prefix to their name. [fschulze]
Add possibility to overwrite the default config name. [tomster]
Improved proxycommand and documented it. [fschulze]
Make the AWS instance available in masters. This changes the get_masters plugin interface. [fschulze]
Use os.execvp instead of subprocess.call. This allows the use of assh in the proxycommand option, which greatly simplifies it’s use. [fschulze]
Added command plugin hooks. [fschulze]
The variable substitution for the proxycommand option now makes the other instances available in a dict under instances. And adds known_hosts. [fschulze]
Load plugins via entry points instead of the plugin section in the config. [fschulze]
Allow fallback to password for ssh to plain instances. [fschulze]
Add option to ask for manual fingerprint validation for plain instances. [fschulze]
0.13 - 2013-09-20
Use os.path.expanduser on all paths, so that one can use ~ in config values like the aws keys. [fschulze]
0.12 - 2013-09-11
There is no need to add the AWS account id to security group names anymore. [fschulze]
Rules are removed from security groups if they aren’t defined in the config. [fschulze]
Allow adding of custom config massagers from inside the config. [fschulze]
Support block device maps to enable use of more than one ephemeral disk. [fschulze]
Added do method on ec2 and plain instances which allows to call fabric commands. [fschulze]
Use PathMassager for access-key-id and secret-access-key in the ec2-master section. This might break existing relative paths for these options. [fschulze]
Added support for EBS boot instances. [fschulze]
Add option ssh-key-filename to point to a private ssh key for ec2 and plain instances. [fschulze]
Fix Fabric integration for newer versions of Fabric. [fschulze]
Support proxycommand option for plain instances. This also caused a change in the init_ssh_key API for plugins. [fschulze]
Support ProxyCommand from ~/.ssh/config for plain instances. Requires Fabric 1.5.0 and Paramiko 1.9.0 or newer. [fschulze]
0.11 - 2012-11-08
Support both the ssh and paramiko libraries depending on which Fabric version is used. [fschulze]
0.10 - 2012-06-04
Added ec2-connection which helps in writing Fabric scripts which don’t connect to a server but need access to the config and AWS (like uploading something to S3). [fschulze]
Fix several problems with using a user name other than root for the do and ssh commands. [fschulze]
Require Fabric >= 1.3.0. [fschulze]
Require boto >= 2.0. [fschulze]
Added hook for startup script options. [fschulze]
Added possibility to configure hooks. [fschulze]
Refactored to enable plugins for different virtualization or cloud providers. [fschulze]
Added lots of tests. [fschulze]
0.9 - 2010-12-09
Overwrites now also affect server creation, not just the startup script. [fschulze]
Added list command which supports just listing snapshots for now. [fschulze]
Added delete-volumes-on-terminate option to delete volumes created from snapshots on instance termination. [fschulze]
Added support for creating volumes from snapshots on instance start. [natea, fschulze]
Added support for ~/.ssh/config. This is a bit limited, because the paramiko config parser isn’t very good. [fschulze]
Added help command which provides some info for zsh autocompletion. [fschulze]
0.8 - 2010-04-21
For the do command the Fabric options reject_unknown_hosts and disable_known_hosts now default to true. [fschulze]
Allow adding normal servers to use with ssh and do commands. [fschulze]
Refactored ssh connection handling to only open network connections when needed. Any fabric option which doesn’t need a connection runs right away now (like -h and -l). [fschulze]
Fix status output after start. [fschulze]
0.7 - 2010-03-22
Added snapshot method to Server class for easy access from fabfiles. [fschulze]
0.6 - 2010-03-18
It’s now possible to specify files which contain the aws keys in the [aws] section with the access-key-id and secret-access-key options. [fschulze]
Added -c/--config option to specify the config file to use. [fschulze]
Added -v/--version option. [tomster (Tom Lazar), fschulze]
Comment lines in the startup script are now removed before any variables in it are expanded, not afterwards. [fschulze]
Use argparse library instead of optparse for more powerful command line parsing. [fschulze]
0.5 - 2010-03-11
Added gzipping of startup script by looking for gzip: prefix in the filename. [fschulze]
Added macro expansion similar to zc.buildout 1.4. [fschulze]
0.4 - 2010-02-18
Check console output in status and tell user about it. [fschulze]
Friendly message instead of traceback when trying to ssh into an unavailable server. [fschulze]
Remove comment lines from startup script if it’s starting with #!/bin/sh or #!/bin/bash. [fschulze]
Removed -r option for start and debug commands and replaced it with more general -o option. [fschulze]
Made startup script optional (not all AMIs support it, especially Windows ones). [fschulze]
The stop command actually only stops an instance now (only works with instances booted from an EBS volume) and the new terminate command now does what stop did before. [fschulze]
Better error message when no console output is available for ssh finger print validation. [fschulze]
Fixed indentation in documentation. [natea (Nate Aune), fschulze]
0.3 - 2010-02-08
Removed the [host_string] prefix of the do command output. [fschulze]
0.2 - 2010-02-02
Snapshots automatically get a description with date and volume id. [fschulze]
The ssh command can now be used with scp and rsync. [fschulze]
0.1 - 2010-01-21
Initial release [fschulze]
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
