Asynchroneous Parallel ssh
Project description
apssh - asynchronous parallel ssh
what's included ?
This tool leverages the Python asynchronous paradigm and comes with
- a standalone
apsshcommand for running remote commands on a bunch of hosts in parallel, as well as theappullandappushcommands to perform parallel copies, that sysadmins might find useful for their routine jobs, - an API with which one can create
SshJobobjects in conjunction with anasynciojob'sSchedulerfor orchestrating them.
Please refer to http://apssh.readthedocs.io/ for a complete documentation bundle.
what is nepi-ng ?
Originally, the idea presented here addresses the needs of experimental research, where an experiment often boils down to running jobs like preparing a set of nodes, initializing them, running some bash script, collecting results, all of them having temporal relationships.
To that end, the 2 libraries asynciojobs and apssh complement each other
to offer a powerful programing environment that we have branded nepi-ng; its 2 parts are thus:
asynciojobs
asynciojobs is a microscopic orchestration scheduler for asyncio-based jobs -
see this link for
details.
This is the part that handles the temporal relationships.
apssh
apssh ships with a few classes that allow you to write jobs in the
asynciojobs sense, that will actually run on ssh:
SshNode: describe how to reach a node (possible through a gateway)SshJob: to run one or several remote commands; each of these can beRun: that is designed to run a command readily available on the target nodeRunScript: when you have a local script file to run remotely, so there is a need to push it over there prior to running itRunString: same idea, but you do not even have a local file, it's just a python string in memory; useful to embed your shell code inside a python codePullorPushfor file transfers over SFTP
As the names may suggest:
- an
SshNodeinstance contains the details of the target node (hostname, username, gateway if relevant, etc...), and it can be thought of as an ssh connection; SshJob, is suitable to run asasynciojobs'sjobs, i.e. inside a scheduler;- an
SshJobinstance contains a list of the actual commands to run, that can be a mix of remote executions and file transfers.
examples
You can see a very simple example of that idea implemented in 2 files
- the python code
- and the related shell script
- plus, a summary of the objects involved is depicted in this figure
Another, simpler example, illustrates how to take advantage of the YAML loader, see
- the
my-first-nepi-ng-scriptfolder in <https://github.com/fit-r2lab/demos - and in particular the 2 versions named
demo-v1.pyanddemo-v2.pythat implement the exact same behaviour but that use- a pure Python approach (v1)
- the YAML loader (v2)
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file apssh-0.25.0.tar.gz.
File metadata
- Download URL: apssh-0.25.0.tar.gz
- Upload date:
- Size: 54.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.9.15
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | fc0cb874f679136ecd1830026826eda680e1fbcfb5594275b27c8a4f8edc5002 |
|
| MD5 | 0634f78c9b9dfff5571d985121241cda |
|
| BLAKE2b-256 | f2b13559f36910a218fc93c1d3b06be56a67762bb5bdc2cbf6c350b4d2b4a5f1 |
File details
Details for the file apssh-0.25.0-py3-none-any.whl.
File metadata
- Download URL: apssh-0.25.0-py3-none-any.whl
- Upload date:
- Size: 50.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.9.15
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | b43a2076572a368f64a5567350013ce588e004047c383ee9bde89e56d913de88 |
|
| MD5 | fa8adf796b214c8fe6a784f67f5c152b |
|
| BLAKE2b-256 | 51c7ff85dfde849f5b601bf3bf1f6c17dc1da5cbf3c4dceb5a036383b2fca932 |
