Execution helpers for simplified usage of subprocess and ssh.

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for penguinolog from gravatar.com penguinolog

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache License, Version 2.0)
  • Author: Alexey Stepanov
  • Tags logging, debugging, development
  • Requires: Python >=2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*
Classifiers

Project description

exec-helpers

https://travis-ci.org/penguinolog/exec-helpers.svg?branch=master https://img.shields.io/appveyor/ci/penguinolog/exec-helpers.svg https://coveralls.io/repos/github/penguinolog/exec-helpers/badge.svg?branch=master Documentation Status https://img.shields.io/pypi/v/exec-helpers.svg https://img.shields.io/pypi/pyversions/exec-helpers.svg https://img.shields.io/pypi/status/exec-helpers.svg https://img.shields.io/github/license/penguinolog/exec-helpers.svg

Execution helpers for simplified usage of subprocess and ssh. Why another subprocess wrapper and why no clear paramiko?

Historically paramiko offers good ssh client, but with specific limitations: you can call command with timeout, but without receiving return code, or call command and wait for return code, but without timeout processing.

In the most cases, we are need just simple SSH client with comfortable API for calls, calls via SSH proxy and checking return code/stderr. This library offers this functionality with connection memorizing, deadlock free polling and friendly result objects (with inline decoding of YAML, JSON, binary or just strings). In addition this library offers the same API for subprocess calls, but with specific limitation: no parallel calls (for protection from race conditions).

Pros:

Python 2.7
Python 3.4
Python 3.5
Python 3.6
PyPy
PyPy3 3.5+

This package includes:

  • SSHClient - historically the first one helper, which used for SSH connections and requires memorization due to impossibility of connection close prediction. Several API calls for sFTP also presents.

  • SSHAuth - class for credentials storage. SSHClient does not store credentials as-is, but uses SSHAuth for it. Objects of this class can be copied between ssh connection objects, also it used for execute_through_host.

  • Subprocess - subprocess.Popen wrapper with timeouts, polling and almost the same API, as SSHClient (except specific flags, like cwd for subprocess and get_tty for ssh).

  • ExecResult - class for execution results storage. Contains exit code, stdout, stderr and getters for decoding as JSON, YAML, string, bytearray and brief strings (up to 7 lines).

  • ExitCodes - enumerator for standard Linux exit codes. BASH return codes (broduced from signal codes) also available.

Usage

SSHClient

Basic initialization of SSHClient can be done without construction of specific objects:

client = exec_helpers.SSHClient(host, username="username", password="password")

If ssh agent is running - keys will be collected by paramiko automatically, but if keys are in specific location - it should be loaded manually and provided as iterable object of paramiko.RSAKey.

For advanced cases or re-use of credentials, SSHAuth object should be used. It can be collected from connection object via property auth.

Creation from scratch:

auth = exec_helpers.SSHAuth(
    username='username',  # type: typing.Optional[str]
    password='password',  # type: typing.Optional[str]
    key=None,  # type: typing.Optional[paramiko.RSAKey]
    keys=None,
)

Key is a main connection key (always tried first) and keys are alternate keys. If main key now correct for username - alternate keys tried, if correct key found - it became main. If no working key - password is used and None is set as main key.

Subprocess

No initialization required.

Base methods

Main methods are execute, check_call and check_stderr for simple executing, executing and checking return code and executing, checking return code and checking for empty stderr output. This methods are almost the same for SSHCleint and Subprocess, except specific flags.

result = helper.execute(
    command,  # type: str
    verbose=False,  # type: bool
    timeout=None,  # type: typing.Optional[int]
    **kwargs
)
result = helper.check_call(
    command,  # type: str
    verbose=False,  # type: bool
    timeout=None,  # type: typing.Optional[int]
    error_info=None,  # type: typing.Optional[str]
    expected=None,  # type: typing.Optional[typing.Iterable[int]]
    raise_on_err=True,  # type: bool
    **kwargs
)
result = helper.check_stderr(
    command,  # type: str
    verbose=False,  # type: bool
    timeout=None,  # type: typing.Optional[int]
    error_info=None,  # type: typing.Optional[str]
    raise_on_err=True,  # type: bool
)

The next command level uses lower level and kwargs are forwarded, so expected exit codes are forwarded from check_stderr. Implementation specific flags are always set via kwargs.

ExecResult

Execution result object has a set of useful properties:

  • cmd - Command

  • exit_code - Command return code. If possible to decode using enumerators for Linux -> it used.

  • stdout -> typing.Tuple[bytes]. Raw stdout output.

  • stderr -> typing.Tuple[bytes]. Raw stderr output.

  • stdout_bin -> bytearray. Binary stdout output.

  • stderr_bin -> bytearray. Binary stderr output.

  • stdout_str -> six.text_types. Text representation of output.

  • stderr_str -> six.text_types. Text representation of output.

  • stdout_brief -> six.text_types. Up to 7 lines from stdout (3 first and 3 last if >7 lines).

  • stderr_brief -> six.text_types. Up to 7 lines from stderr (3 first and 3 last if >7 lines).

  • stdout_json - STDOUT decoded as JSON.

  • stdout_yaml - STDOUT decoded as YAML.

  • timestamp -> typing.Optional(datetime.datetime). Timestamp for received exit code.

SSHClient specific

SSHClient commands support get_pty flag, which enables PTY open on remote side. PTY width and height can be set via kwargs, dimensions in pixels are always 0x0.

Possible to call commands in parallel on multiple hosts if it’s not produce huge output:

results = SSHClient.execute_together(
    remotes,  # type: typing.Iterable[SSHClient]
    command,  # type: str
    timeout=None,  # type: typing.Optional[int]
    expected=None,  # type: typing.Optional[typing.Iterable[int]]
    raise_on_err=True  # type: bool
)
results  # type: typing.Dict[typing.Tuple[str, int], exec_result.ExecResult]

Results is a dict with keys = (hostname, port) and and results in values. By default execute_together raises exception if unexpected return code on any remote.

For execute through SSH host can be used execute_through_host method:

result = client.execute_through_host(
    hostname,  # type: str
    command,  # type: str
    auth=None,  # type: typing.Optional[SSHAuth]
    target_port=22,  # type: int
    timeout=None,  # type: typing.Optional[int]
    verbose=False,  # type: bool
    get_pty=False,  # type: bool
)

Where hostname is a target hostname, auth is an alternate credentials for target host.

SSH client implements fast sudo support via context manager: Commands will be run with sudo enforced independently from client settings for normal usage:

with client.sudo(enforce=True):
    ...

Commands will be run without sudo independently from client settings for normal usage:

with client.sudo(enforce=False):
    ...

“Permanent client setting”:

client.sudo_mode = mode  # where mode is True or False

SSH Client supports sFTP for working with remote files:

with client.open(path, mode='r') as f:
    ...

For fast remote paths checks available methods:

  • exists(path) -> bool

>>> conn.exists('/etc/passwd')
True

  • stat(path) -> paramiko.sftp_attr.SFTPAttributes

>>> conn.stat('/etc/passwd')
<SFTPAttributes: [ size=1882 uid=0 gid=0 mode=0o100644 atime=1521618061 mtime=1449733241 ]>
>>> str(conn.stat('/etc/passwd'))
'-rw-r--r--   1 0        0            1882 10 Dec 2015  ?'

  • isfile(path) -> bool

>>> conn.isfile('/etc/passwd')
True

  • isdir(path) -> bool

>>> conn.isdir('/etc/passwd')
False

Additional (non-standard) helpers:

  • mkdir(path: str) - execute mkdir -p path

  • rm_rf(path: str) - execute rm -rf path

  • upload(source: str, target: str) - upload file or from source to target using sFTP.

  • download(destination: str, target: str) - download file from target to destination using sFTP.

Subprocess specific

Kwargs set properties:

  • cwd - working directory.

  • env - environment variables dict.

Testing

The main test mechanism for the package exec-helpers is using tox. Test environments available:

pep8
py27
py34
py35
py36
pypy
pypy3
pylint
pep257

CI systems

For code checking several CI systems is used in parallel:

  1. Travis CI: is used for checking: PEP8, pylint, bandit, installation possibility and unit tests. Also it’s publishes coverage on coveralls.

  2. AppVeyor: is used for checking windows compatibility.

  3. coveralls: is used for coverage display.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for penguinolog from gravatar.com penguinolog

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache License, Version 2.0)
  • Author: Alexey Stepanov
  • Tags logging, debugging, development
  • Requires: Python >=2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*
Classifiers

Release history Release notifications | RSS feed

8.1.2

8.1.1

8.1.0

8.0.1

8.0.0.post0

8.0.0

7.4.0

7.3.6

7.3.5.post0

7.3.4

7.3.3

7.3.2

7.3.1

7.3.0

7.2.1

7.2.0

7.1.4.post0

7.1.4

7.1.3

7.1.1

7.1.0.post0

7.1.0

7.0.5

7.0.4

7.0.3

7.0.2

7.0.1

7.0.0

6.1.6.post0

6.1.5

6.1.4

6.1.3

6.1.2

6.1.1

6.1.0

6.0.0

6.0.0rc3 pre-release

6.0.0rc2 pre-release

6.0.0rc1 pre-release

6.0.0b6 pre-release

6.0.0b5 pre-release

6.0.0b4 pre-release

6.0.0b3 pre-release

6.0.0b2 pre-release

6.0.0b1 pre-release

6.0.0a2 pre-release

6.0.0a1 pre-release

5.2.4

5.2.3

5.2.2

5.2.0

5.1.1

5.1.0

5.0.0

4.4.0

4.3.1

4.3.0

4.2.0

4.1.7

4.1.6

4.1.5

4.1.4

4.1.3

4.1.2

4.1.1

4.1.0

4.0.1

4.0.0

3.7.0

3.6.0

3.5.7

3.5.5

3.5.4

3.5.3

3.5.2

3.5.1

3.5.0

3.4.0.post2

3.4.0.post1

3.4.0

3.3.2

3.3.0

3.2.1

3.2.0

3.1.5

3.1.4

3.1.3

3.1.2

3.1.1

3.1.0

3.0.0

2.14.0

2.13.0

2.12.2

2.12.1

2.12.0

2.11.3

2.11.2

2.11.1

2.11.0

2.10.0

2.9.6

2.9.4

2.9.3

2.9.2

2.9.1

2.9.0

2.2.0

2.1.3

2.1.2

2.1.1

2.1.0

2.0.2

2.0.1

2.0.0

1.14.3

1.14.2

1.13.0

1.12.2

1.12.1

1.12.0

1.11.1

1.11.0

1.10.0

1.9.9

1.9.7

1.9.6

1.9.5

1.9.4

1.9.3

1.9.2

1.9.1

1.9.0

1.4.1

1.4.0

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.2

1.2.1

1.2.0

1.1.2

1.1.1

1.1.0

This version

1.0.0

0.9.9

0.9.0

0.8.3

0.8.2

0.8.1

0.8.0

0.7.2

0.7.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

exec-helpers-1.0.0.tar.gz (490.2 kB view details)

Uploaded Source

Built Distributions

exec_helpers-1.0.0-cp36-none-win_amd64.whl (373.0 kB view details)

Uploaded CPython 3.6 Windows x86-64

exec_helpers-1.0.0-cp36-none-win32.whl (324.6 kB view details)

Uploaded CPython 3.6 Windows x86

exec_helpers-1.0.0-cp36-cp36m-manylinux1_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.6m

exec_helpers-1.0.0-cp36-cp36m-manylinux1_i686.whl (1.5 MB view details)

Uploaded CPython 3.6m

exec_helpers-1.0.0-cp35-none-win_amd64.whl (363.9 kB view details)

Uploaded CPython 3.5 Windows x86-64

exec_helpers-1.0.0-cp35-none-win32.whl (318.7 kB view details)

Uploaded CPython 3.5 Windows x86

exec_helpers-1.0.0-cp35-cp35m-manylinux1_x86_64.whl (1.6 MB view details)

Uploaded CPython 3.5m

exec_helpers-1.0.0-cp35-cp35m-manylinux1_i686.whl (1.5 MB view details)

Uploaded CPython 3.5m

exec_helpers-1.0.0-cp34-none-win_amd64.whl (356.3 kB view details)

Uploaded CPython 3.4 Windows x86-64

exec_helpers-1.0.0-cp34-none-win32.whl (323.9 kB view details)

Uploaded CPython 3.4 Windows x86

exec_helpers-1.0.0-cp34-cp34m-manylinux1_x86_64.whl (1.6 MB view details)

Uploaded CPython 3.4m

exec_helpers-1.0.0-cp34-cp34m-manylinux1_i686.whl (1.5 MB view details)

Uploaded CPython 3.4m

exec_helpers-1.0.0-cp27-none-win_amd64.whl (368.4 kB view details)

Uploaded CPython 2.7 Windows x86-64

exec_helpers-1.0.0-cp27-none-win32.whl (324.5 kB view details)

Uploaded CPython 2.7 Windows x86

exec_helpers-1.0.0-cp27-cp27mu-manylinux1_x86_64.whl (1.5 MB view details)

Uploaded CPython 2.7mu

exec_helpers-1.0.0-cp27-cp27mu-manylinux1_i686.whl (1.3 MB view details)

Uploaded CPython 2.7mu

File details

Details for the file exec-helpers-1.0.0.tar.gz.

File metadata

File hashes

Hashes for exec-helpers-1.0.0.tar.gz
Algorithm Hash digest
SHA256 7e09511199b76edfa134407c3bd4619f74bd6dd244f38619ad0c082ece354690
MD5 788488ac2bc8e58919f457d4ef006f7f
BLAKE2b-256 ea8e0f4d4c61693409e4442090d55a34e1a7cd8755236679616a018c0f6d6a0b

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp36-none-win_amd64.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp36-none-win_amd64.whl
Algorithm Hash digest
SHA256 c1aab1b6ca3faf654feb20a250592f3d9cf72739193730be871a4637092b871b
MD5 23a22602d1c360c88dfe50e911c45090
BLAKE2b-256 91637361a933d3b77d6eed5d3c36c8b26baf8d4ddd3a5bd1a5681548153f5355

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp36-none-win32.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp36-none-win32.whl
Algorithm Hash digest
SHA256 b011a4261c829245834763636617efa687aaa2f23be4932af339af6a1efbfe84
MD5 b3de56d93f52c7a4addf7f419fd55ee7
BLAKE2b-256 c3305efa674a0e6b65258fb932f1f16532fb4d41ad82049941466e4dfea6da66

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp36-cp36m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp36-cp36m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 203cf84a5ba5beb52f1e8856cab603bcf3fe15cedc0527022aec46bf6dbffbb5
MD5 1aa656bfc99615ab392d2ea2cb5954f5
BLAKE2b-256 cc1e55b07c4336dd5d9d7e9bc04753d02f46f91b5833e2bd5f4dd9f4d5188b51

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp36-cp36m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp36-cp36m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 80ebe6ef3cdb80a6fa04f48209460987e16c41bb6a1e91d28e9f69963a498e26
MD5 d53fe9a70354665caef6301fe3580698
BLAKE2b-256 a8c19e5a7f365cb105ef87760cc67ba9e0714561f6da51a409a33f7a285ae361

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp35-none-win_amd64.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp35-none-win_amd64.whl
Algorithm Hash digest
SHA256 f89cd61dd5b4e28731903aacca8845a7be69bab1466aea285630d6eacc69807b
MD5 3d19a454ed47222251fb4732c70b157c
BLAKE2b-256 fb81c4b2a00e33146bb67ea98481a33e05321e1c995e0f8b9e4ccb425a7f5c86

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp35-none-win32.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp35-none-win32.whl
Algorithm Hash digest
SHA256 c7df132301343fe01c9b28443386c9ee312a9b3644b70ae70f2f0add2272e04f
MD5 89b0417cec4458c3f985381b9ebb2618
BLAKE2b-256 c4c964bacf6f8a1bdad81aa1e087fd5a4aa550fb1edcd3549fa2b91d77bdc58e

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp35-cp35m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp35-cp35m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 d4f740c6bf501b66e12601162287ddaee8e1b901ee7b71fae6e4bd09dd8922cc
MD5 e73561cbca4d4e83bc0b7693e3dc1c15
BLAKE2b-256 bbd912e175c768265cfbe05b98f830f52f55d0aba47a13204d57f56e9041cc78

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp35-cp35m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp35-cp35m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 7f3928fa3ae80691f4470d1f8590134fa37e5a720556483460b9c02510f2259b
MD5 f9f44a773e9a24d09055f227a55adc05
BLAKE2b-256 b45eaad898dea275d86cbd41dee2f54ad408a78f244a8b49791bfc402fac9606

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp34-none-win_amd64.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp34-none-win_amd64.whl
Algorithm Hash digest
SHA256 09b2ca0df7c3403503e7d666a60bbb4e92aee19c578e6c74b627b4d23021ebbe
MD5 00936a1c3b55789d340df63730010c87
BLAKE2b-256 668c16fae314e1d7ee2f53b797ed96f8b72f9b3065860388960dc5be393c9b16

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp34-none-win32.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp34-none-win32.whl
Algorithm Hash digest
SHA256 982d542647a97608a39de8a30d51ed7f132cf9d2c6f492c8ae88c9358a75ec89
MD5 6846709c7eeaa1fd45dfe996739fd889
BLAKE2b-256 2c6f82f3674f87480a1d42289ab9b86667571272f8c2fc61480ae36106b6c810

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp34-cp34m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp34-cp34m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 44eb7c7ee64ce3c801acfa7fa147a17b701673300839e0d77667c37b5c857b5a
MD5 b5c421331418bec95370442c47b3aecc
BLAKE2b-256 c2bf943502f201934df76de8573dfa1b552a1fc43f8fc7efa9e83ce46a84789b

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp34-cp34m-manylinux1_i686.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp34-cp34m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 a2da43a267eb0a7c3d75ee2929388e1f04f7044363c973ab6b13080cad98d1d5
MD5 0a94504cb2c3674d62eceac817b5370c
BLAKE2b-256 79a2345ea0dd7fbd0b2a256b6289eacc3ad65ce9f9369e266dff2b5a5a83e2f5

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp27-none-win_amd64.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp27-none-win_amd64.whl
Algorithm Hash digest
SHA256 acc5c4d20e1528e9035310c72479c50bd486fa141aed2254b9e5a1cdab84987f
MD5 deda5d84354f87f7bf2ede6a957e21a1
BLAKE2b-256 d66b50958850279b0873c211a85264671cbfc1e3d8b52a6853d87e1ecad20470

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp27-none-win32.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp27-none-win32.whl
Algorithm Hash digest
SHA256 9359419b8f4aa664aee42cb07b7e3b07ff5b58bb9ed27e1da5582086bd1df193
MD5 fca452391288af929cef10c14b023911
BLAKE2b-256 a5a2c0a7ef62d07eeed9857a2327549542c3fd45d2562d1f5a58aaff1dbb20c0

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp27-cp27mu-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 e20b7e26ac70a4e4888c5c9732482abae068fe18cbb61e731aba193c3e7258ee
MD5 5e09160c53058449cc6aa99e9c23fd92
BLAKE2b-256 55888218fb1c4c09e53200b2fc10b90073bc9b498c4104d539e4fa81fedf15a3

See more details on using hashes here.

File details

Details for the file exec_helpers-1.0.0-cp27-cp27mu-manylinux1_i686.whl.

File metadata

File hashes

Hashes for exec_helpers-1.0.0-cp27-cp27mu-manylinux1_i686.whl
Algorithm Hash digest
SHA256 1dc174973743fd00d76c0969c439fe962c0b1882daafa83aaa2af63cabb23424
MD5 3ebc3dc8d70bfaa0104b390c4f0178cc
BLAKE2b-256 99b59acb374e1e77c95edb7d3a69aa0bd68aeca357c4aa1beb8ca9b067f79622

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page