Execution helpers for simplified usage of subprocess and ssh.

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for penguinolog from gravatar.com penguinolog

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache License, Version 2.0)
  • Author: Alexey Stepanov
  • Tags logging, debugging, development
  • Requires: Python >=2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*
Classifiers

Project description

exec-helpers

https://travis-ci.org/penguinolog/exec-helpers.svg?branch=master https://img.shields.io/appveyor/ci/penguinolog/exec-helpers.svg https://coveralls.io/repos/github/penguinolog/exec-helpers/badge.svg?branch=master Documentation Status https://img.shields.io/pypi/v/exec-helpers.svg https://img.shields.io/pypi/pyversions/exec-helpers.svg https://img.shields.io/pypi/status/exec-helpers.svg https://img.shields.io/github/license/penguinolog/exec-helpers.svg

Execution helpers for simplified usage of subprocess and ssh. Why another subprocess wrapper and why no clear paramiko?

Historically paramiko offers good ssh client, but with specific limitations: you can call command with timeout, but without receiving return code, or call command and wait for return code, but without timeout processing.

In the most cases, we are need just simple SSH client with comfortable API for calls, calls via SSH proxy and checking return code/stderr. This library offers this functionality with connection memorizing, deadlock free polling and friendly result objects (with inline decoding of YAML, JSON, binary or just strings). In addition this library offers the same API for subprocess calls, but with specific limitation: no parallel calls (for protection from race conditions).

Pros:

Python 2.7
Python 3.4
Python 3.5
Python 3.6
PyPy
PyPy3 3.5+

This package includes:

  • SSHClient - historically the first one helper, which used for SSH connections and requires memorization due to impossibility of connection close prediction. Several API calls for sFTP also presents.

  • SSHAuth - class for credentials storage. SSHClient does not store credentials as-is, but uses SSHAuth for it. Objects of this class can be copied between ssh connection objects, also it used for execute_through_host.

  • Subprocess - subprocess.Popen wrapper with timeouts, polling and almost the same API, as SSHClient (except specific flags, like cwd for subprocess and get_tty for ssh).

  • ExecResult - class for execution results storage. Contains exit code, stdout, stderr and getters for decoding as JSON, YAML, string, bytearray and brief strings (up to 7 lines).

  • ExitCodes - enumerator for standard Linux exit codes. BASH return codes (broduced from signal codes) also available.

Usage

SSHClient

Basic initialization of SSHClient can be done without construction of specific objects:

client = exec_helpers.SSHClient(host, username="username", password="password")

If ssh agent is running - keys will be collected by paramiko automatically, but if keys are in specific location - it should be loaded manually and provided as iterable object of paramiko.RSAKey.

For advanced cases or re-use of credentials, SSHAuth object should be used. It can be collected from connection object via property auth.

Creation from scratch:

auth = exec_helpers.SSHAuth(
    username='username',  # type: typing.Optional[str]
    password='password',  # type: typing.Optional[str]
    key=None,  # type: typing.Optional[paramiko.RSAKey]
    keys=None,
)

Key is a main connection key (always tried first) and keys are alternate keys. If main key now correct for username - alternate keys tried, if correct key found - it became main. If no working key - password is used and None is set as main key.

Subprocess

No initialization required.

Base methods

Main methods are execute, check_call and check_stderr for simple executing, executing and checking return code and executing, checking return code and checking for empty stderr output. This methods are almost the same for SSHCleint and Subprocess, except specific flags.

result = helper.execute(
    command,  # type: str
    verbose=False,  # type: bool
    timeout=None,  # type: typing.Optional[int]
    **kwargs
)
result = helper.check_call(
    command,  # type: str
    verbose=False,  # type: bool
    timeout=None,  # type: typing.Optional[int]
    error_info=None,  # type: typing.Optional[str]
    expected=None,  # type: typing.Optional[typing.Iterable[int]]
    raise_on_err=True,  # type: bool
    **kwargs
)
result = helper.check_stderr(
    command,  # type: str
    verbose=False,  # type: bool
    timeout=None,  # type: typing.Optional[int]
    error_info=None,  # type: typing.Optional[str]
    raise_on_err=True,  # type: bool
)

The next command level uses lower level and kwargs are forwarded, so expected exit codes are forwarded from check_stderr. Implementation specific flags are always set via kwargs.

ExecResult

Execution result object has a set of useful properties:

  • cmd - Command

  • exit_code - Command return code. If possible to decode using enumerators for Linux -> it used.

  • stdout -> typing.Tuple[bytes]. Raw stdout output.

  • stderr -> typing.Tuple[bytes]. Raw stderr output.

  • stdout_bin -> bytearray. Binary stdout output.

  • stderr_bin -> bytearray. Binary stderr output.

  • stdout_str -> six.text_types. Text representation of output.

  • stderr_str -> six.text_types. Text representation of output.

  • stdout_brief -> six.text_types. Up to 7 lines from stdout (3 first and 3 last if >7 lines).

  • stderr_brief -> six.text_types. Up to 7 lines from stderr (3 first and 3 last if >7 lines).

  • stdout_json - STDOUT decoded as JSON.

  • stdout_yaml - STDOUT decoded as YAML.

  • timestamp -> typing.Optional(datetime.datetime). Timestamp for received exit code.

SSHClient specific

SSHClient commands support get_pty flag, which enables PTY open on remote side. PTY width and height can be set via kwargs, dimensions in pixels are always 0x0.

Possible to call commands in parallel on multiple hosts if it’s not produce huge output:

results = SSHClient.execute_together(
    remotes,  # type: typing.Iterable[SSHClient]
    command,  # type: str
    timeout=None,  # type: typing.Optional[int]
    expected=None,  # type: typing.Optional[typing.Iterable[int]]
    raise_on_err=True  # type: bool
)
results  # type: typing.Dict[typing.Tuple[str, int], exec_result.ExecResult]

Results is a dict with keys = (hostname, port) and and results in values. By default execute_together raises exception if unexpected return code on any remote.

For execute through SSH host can be used execute_through_host method:

result = client.execute_through_host(
    hostname,  # type: str
    command,  # type: str
    auth=None,  # type: typing.Optional[SSHAuth]
    target_port=22,  # type: int
    timeout=None,  # type: typing.Optional[int]
    verbose=False,  # type: bool
    get_pty=False,  # type: bool
)

Where hostname is a target hostname, auth is an alternate credentials for target host.

SSH client implements fast sudo support via context manager: Commands will be run with sudo enforced independently from client settings for normal usage:

with client.sudo(enforce=True):
    ...

Commands will be run without sudo independently from client settings for normal usage:

with client.sudo(enforce=False):
    ...

“Permanent client setting”:

client.sudo_mode = mode  # where mode is True or False

SSH Client supports sFTP for working with remote files:

with client.open(path, mode='r') as f:
    ...

For fast remote paths checks available methods:

  • exists(path) -> bool

>>> conn.exists('/etc/passwd')
True

  • stat(path) -> paramiko.sftp_attr.SFTPAttributes

>>> conn.stat('/etc/passwd')
<SFTPAttributes: [ size=1882 uid=0 gid=0 mode=0o100644 atime=1521618061 mtime=1449733241 ]>
>>> str(conn.stat('/etc/passwd'))
'-rw-r--r--   1 0        0            1882 10 Dec 2015  ?'

  • isfile(path) -> bool

>>> conn.isfile('/etc/passwd')
True

  • isdir(path) -> bool

>>> conn.isdir('/etc/passwd')
False

Additional (non-standard) helpers:

  • mkdir(path: str) - execute mkdir -p path

  • rm_rf(path: str) - execute rm -rf path

  • upload(source: str, target: str) - upload file or from source to target using sFTP.

  • download(destination: str, target: str) - download file from target to destination using sFTP.

Subprocess specific

Kwargs set properties:

  • cwd - working directory.

  • env - environment variables dict.

Testing

The main test mechanism for the package exec-helpers is using tox. Test environments available:

pep8
py27
py34
py35
py36
pypy
pypy3
pylint
pep257

CI systems

For code checking several CI systems is used in parallel:

  1. Travis CI: is used for checking: PEP8, pylint, bandit, installation possibility and unit tests. Also it’s publishes coverage on coveralls.

  2. AppVeyor: is used for checking windows compatibility.

  3. coveralls: is used for coverage display.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for penguinolog from gravatar.com penguinolog

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache License, Version 2.0)
  • Author: Alexey Stepanov
  • Tags logging, debugging, development
  • Requires: Python >=2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*
Classifiers

Release history Release notifications | RSS feed

8.1.2

8.1.1

8.1.0

8.0.1

8.0.0.post0

8.0.0

7.4.0

7.3.6

7.3.5.post0

7.3.4

7.3.3

7.3.2

7.3.1

7.3.0

7.2.1

7.2.0

7.1.4.post0

7.1.4

7.1.3

7.1.1

7.1.0.post0

7.1.0

7.0.5

7.0.4

7.0.3

7.0.2

7.0.1

7.0.0

6.1.6.post0

6.1.5

6.1.4

6.1.3

6.1.2

6.1.1

6.1.0

6.0.0

6.0.0rc3 pre-release

6.0.0rc2 pre-release

6.0.0rc1 pre-release

6.0.0b6 pre-release

6.0.0b5 pre-release

6.0.0b4 pre-release

6.0.0b3 pre-release

6.0.0b2 pre-release

6.0.0b1 pre-release

6.0.0a2 pre-release

6.0.0a1 pre-release

5.2.4

5.2.3

5.2.2

5.2.0

5.1.1

5.1.0

5.0.0

4.4.0

4.3.1

4.3.0

4.2.0

4.1.7

4.1.6

4.1.5

4.1.4

4.1.3

4.1.2

4.1.1

4.1.0

4.0.1

4.0.0

3.7.0

3.6.0

3.5.7

3.5.5

3.5.4

3.5.3

3.5.2

3.5.1

3.5.0

3.4.0.post2

3.4.0.post1

3.4.0

3.3.2

3.3.0

3.2.1

3.2.0

3.1.5

3.1.4

3.1.3

3.1.2

3.1.1

3.1.0

3.0.0

2.14.0

2.13.0

2.12.2

2.12.1

2.12.0

2.11.3

2.11.2

2.11.1

2.11.0

2.10.0

2.9.6

2.9.4

2.9.3

2.9.2

2.9.1

2.9.0

2.2.0

2.1.3

2.1.2

2.1.1

2.1.0

2.0.2

2.0.1

2.0.0

1.14.3

1.14.2

1.13.0

1.12.2

1.12.1

1.12.0

1.11.1

1.11.0

1.10.0

1.9.9

1.9.7

1.9.6

1.9.5

1.9.4

1.9.3

1.9.2

1.9.1

1.9.0

1.4.1

1.4.0

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.2

1.2.1

1.2.0

1.1.2

1.1.1

1.1.0

1.0.0

This version

0.9.9

0.9.0

0.8.3

0.8.2

0.8.1

0.8.0

0.7.2

0.7.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

exec-helpers-0.9.9.tar.gz (488.5 kB view details)

Uploaded Source

Built Distributions

exec_helpers-0.9.9-cp36-none-win_amd64.whl (370.5 kB view details)

Uploaded CPython 3.6 Windows x86-64

exec_helpers-0.9.9-cp36-none-win32.whl (323.0 kB view details)

Uploaded CPython 3.6 Windows x86

exec_helpers-0.9.9-cp36-cp36m-manylinux1_x86_64.whl (1.6 MB view details)

Uploaded CPython 3.6m

exec_helpers-0.9.9-cp36-cp36m-manylinux1_i686.whl (1.5 MB view details)

Uploaded CPython 3.6m

exec_helpers-0.9.9-cp35-none-win_amd64.whl (362.6 kB view details)

Uploaded CPython 3.5 Windows x86-64

exec_helpers-0.9.9-cp35-none-win32.whl (317.4 kB view details)

Uploaded CPython 3.5 Windows x86

exec_helpers-0.9.9-cp35-cp35m-manylinux1_x86_64.whl (1.6 MB view details)

Uploaded CPython 3.5m

exec_helpers-0.9.9-cp35-cp35m-manylinux1_i686.whl (1.5 MB view details)

Uploaded CPython 3.5m

exec_helpers-0.9.9-cp34-none-win_amd64.whl (354.3 kB view details)

Uploaded CPython 3.4 Windows x86-64

exec_helpers-0.9.9-cp34-none-win32.whl (322.3 kB view details)

Uploaded CPython 3.4 Windows x86

exec_helpers-0.9.9-cp34-cp34m-manylinux1_x86_64.whl (1.6 MB view details)

Uploaded CPython 3.4m

exec_helpers-0.9.9-cp34-cp34m-manylinux1_i686.whl (1.5 MB view details)

Uploaded CPython 3.4m

exec_helpers-0.9.9-cp27-none-win_amd64.whl (366.1 kB view details)

Uploaded CPython 2.7 Windows x86-64

exec_helpers-0.9.9-cp27-none-win32.whl (322.8 kB view details)

Uploaded CPython 2.7 Windows x86

exec_helpers-0.9.9-cp27-cp27mu-manylinux1_x86_64.whl (1.5 MB view details)

Uploaded CPython 2.7mu

exec_helpers-0.9.9-cp27-cp27mu-manylinux1_i686.whl (1.3 MB view details)

Uploaded CPython 2.7mu

File details

Details for the file exec-helpers-0.9.9.tar.gz.

File metadata

File hashes

Hashes for exec-helpers-0.9.9.tar.gz
Algorithm Hash digest
SHA256 df35e2ce7f0b97696142ade822446633115436428678d7e10c4c0b5faf15af4e
MD5 2dfaff14dce4e12ab5daa73e28471588
BLAKE2b-256 c1ca0d4e8114321134947fd902775bfb88c76fc750f0f27e61f6b1285263e0e7

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp36-none-win_amd64.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp36-none-win_amd64.whl
Algorithm Hash digest
SHA256 6bc7c4928c237fba13899225e50813c36cdb1a4dab59d7006dfed1d8f03a6317
MD5 bd4526a0a096a124081aa4d710611728
BLAKE2b-256 ee5581d9e93a5e4af0a10560b4208b09ad46dae89c39677709edcb3da57aa4de

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp36-none-win32.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp36-none-win32.whl
Algorithm Hash digest
SHA256 dceb6ef5c1a974231e998313f973a6140dcdb2eb6c1551ac7f324fbc7883949f
MD5 4a566bbcb63486e6afffb409f31681c7
BLAKE2b-256 881609e1aa7c8fdd8b26874d0c615e77d01d885e413a8826884fcefe44662225

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp36-cp36m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp36-cp36m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 6ccdf687dd9a1c0ac94d96a5ffb44d36c747c41c0818f968fc79d8e68def0ca9
MD5 530b3bb9c4894ed7dd459d8cfaedfe83
BLAKE2b-256 76fc41a1e5a017ed35bfb32da78514934ea4a3e482a74a3ca8af9b29b195dfd9

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp36-cp36m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp36-cp36m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 3651bb7adb7ee28600fb93a8a69558ecde9165688831455a00ca253bd7a1e37d
MD5 afc6104dfe77dcd5f67de45f6be00507
BLAKE2b-256 5c798ea370ff732dd3ea8b58bb0907d353b549574f57bad027067d46deba9bf6

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp35-none-win_amd64.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp35-none-win_amd64.whl
Algorithm Hash digest
SHA256 8479aeab9b93fc33b95a3b4b4eb0fca739de8ba4a59b5d3d3997bce99eb4111b
MD5 7cfe302879d8ea94cf4b9a0bf92f7643
BLAKE2b-256 054df627e81e4321b109e84978c9e39d7a7a45af138616d4b4283095e4436f0a

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp35-none-win32.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp35-none-win32.whl
Algorithm Hash digest
SHA256 b022b181cce0a8b1f2b6f27ab1f8d4f870faf701a6db89aa93f1a51037b07112
MD5 5b19b5256a08e9c6666ab320514d74ff
BLAKE2b-256 e47d8e6e181e36fef52e8d24ab25354eee3342db02249d50483d9192446595fc

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp35-cp35m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp35-cp35m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 ba43e9b4885b639bad269c2997a35a9351930e0a39baaa6a7a696ecc7d04e5c5
MD5 56da81773ba3d1027025ec09a77bf054
BLAKE2b-256 e7dab1a9365e3d0871fedfa7674c249de6ea2d6401e4770582d30ce9c3c1ceba

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp35-cp35m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp35-cp35m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 ee59f191bc160fae0a2c78d28968a44e8c28c0e6b12a78560de7f3f2004387b3
MD5 6930e4d3cb9d3459451606cffdb9ee25
BLAKE2b-256 61e1010bc5e1811270d108d7ef2cb254682bccae3b70bbbf8813a6f4007af87a

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp34-none-win_amd64.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp34-none-win_amd64.whl
Algorithm Hash digest
SHA256 5061e5aa685782b7cd10494ec6c0b5ee78966bfe2e48ea59586ac19e99b39df2
MD5 b4312834b9f00d50d62fbb8754c2e04d
BLAKE2b-256 60e0165b0f2bbef4fceb8c3e1ea0483950d4640ff3cd5b47ef0463119751d3df

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp34-none-win32.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp34-none-win32.whl
Algorithm Hash digest
SHA256 6b12006adeabbc1e443f9a6e782ff60247ab0b3607e90be67847b247e1487927
MD5 a8b89b9be90434b73e54f8576f42bb38
BLAKE2b-256 d08a16caf074bde626b2ac3f795dd9adf002344ee997cdef676e45ec6e5807e8

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp34-cp34m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp34-cp34m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 f648f85b144db7e76885ca450faa7f6270bc0dcdeb3a1b6d09b2bf3d29eb7d6e
MD5 609da2332c15f9170f1cbe2a7cf3e89d
BLAKE2b-256 6dd622e86066761d2a291e5fcabe0bee95ebadb55146ed6cd6a4030a2384ce57

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp34-cp34m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp34-cp34m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 d8305160cbe9f17b2251583320b0360fc681e91c4ca9230b3d559372511c8614
MD5 99a793524c3640e0eef57b1d901c9c26
BLAKE2b-256 fe565cffd79dcea8278ca048bafcc9b7e183b77ced6ab2dce5fbc6849a49df79

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp27-none-win_amd64.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp27-none-win_amd64.whl
Algorithm Hash digest
SHA256 6680e9c48ccd9739f95bcce40d76164d5c6c135e6fc9ba1d1a4be9b6179f21af
MD5 902e2272ef65c967cc51db1cae9858ef
BLAKE2b-256 fdf416f0c26185e3683fc97099a22ee7f1a7c779fdb1e7fb13f12711ab2fd019

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp27-none-win32.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp27-none-win32.whl
Algorithm Hash digest
SHA256 9d0afe179075c829b15682720ef673ee0bbd0fee79b28587256b1d450309ff97
MD5 220e911f4382602d2a38aaf4948fae2f
BLAKE2b-256 edc0109d225c191c64c5c3c061a6545a2f248c5a16e69a9fcd9edd148f08ab24

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp27-cp27mu-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 624cca2b4893385cfab3de1a580ae55bf57f1f4807a66f4d10196791f38c06f8
MD5 a09afce424e857e3637216e02521e9c2
BLAKE2b-256 dea6827adead8b2605c86c6ff9a73cae6b8969f44512dc67d187ca384e185609

See more details on using hashes here.

File details

Details for the file exec_helpers-0.9.9-cp27-cp27mu-manylinux1_i686.whl.

File metadata

File hashes

Hashes for exec_helpers-0.9.9-cp27-cp27mu-manylinux1_i686.whl
Algorithm Hash digest
SHA256 fba659c0c3f3c82aa364c601475024b75077c59478d0a59f074d3854bfb68e6c
MD5 fba1673810c1bb7a93fa6f2cf82e13af
BLAKE2b-256 50211646f9a885fb9b270a0ef3acf06bd6172015aee219f87316de9e9b4359e1

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page