Skip to main content

Beautiful Async Subprocess executor

Project description

Why?

In Python we now have async subprocesses which allows to execute several subprocesses at the same time. The purpose of this library is to:

  • provide an acceptable asyncio subprocess wrapper for my syntaxic taste,

  • can stream stderr and stdout in real time while capturing it,

  • real time output must be prefixed for when you execute several commands at the time so that you know which line is for which process, like with docker-compose logs,

  • output coloration in real time with regexps to make even more readable.

This code was copy/pasted between projects and finally extracted on its own.

Usage

Basics

Basic example, this will both stream output and capture it:

from shlax import Subprocess
proc = await Subprocess('echo hi').wait()
print(proc.rc, proc.out, proc.err, proc.out_raw, proc.err_raw)

Longer

If you want to start the command and wait for completion elsewhere then call any of start() and wait(), or both, explicitely:

proc = Subprocess('echo hi')
await proc.start()  # start the process
await proc.wait()   # wait for completion

Proc alias

Note that shlax defines an alias Proc to Subprocess so this also works:

from shlax import Proc
proc = await Proc('echo hi').wait()

Quiet

To disable real time output streaming use the quiet argument:

proc = await Subprocess('echo hi', quiet=True).wait()

Prefix

Using prefixes, you can have real time outputs of parallel commands and at the same time know which output belongs to which process:

proc0 = Subprocess('find /', prefix='first')
proc1 = Subprocess('find /', prefix='second')
await asyncio.gather(proc0.wait(), proc1.wait())

Coloration and output patching

You can add coloration or patch real time output with regexps, note that it will be applied line by line:

import sys
regexps = {
    '^(.*).py$': '{cyan}\\1',
}
await asyncio.gather(*[
    Subprocess(
        f'find {path}',
        regexps=regexps,
    ).wait()
    for path in sys.path
])

Where is the rest?

Shlax used to be the name of a much more ambitious poc-project that has been extracted in two projects with clear boundaries, namely sysplan and podplan which are still in alpha state, but Shlax as it is feature complete and stable.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

shlax-1.0.0.tar.gz (4.3 kB view details)

Uploaded Source

File details

Details for the file shlax-1.0.0.tar.gz.

File metadata

  • Download URL: shlax-1.0.0.tar.gz
  • Upload date:
  • Size: 4.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for shlax-1.0.0.tar.gz
Algorithm Hash digest
SHA256 a46248137af85fdec9be67fefc600765c93d031786ec9d6926f45a49871d234f
MD5 13cfe75121b8ebdf387fc5b1e73653d7
BLAKE2b-256 82d60afab025caa636ba82576319141ddd0c6f6f7f1d7bfb75324866e4e59547

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page