Skip to main content

Async-related stuff you miss in standard library

Project description

Async-related stuff you miss in standard library

https://upload.wikimedia.org/wikipedia/commons/thumb/f/fe/The_Hanging_by_Jacques_Callot.jpg/500px-The_Hanging_by_Jacques_Callot.jpg

“The horrors of hanging tasks when using asyncio” by Jacques Callot

Safely run coroutines concurrently

The asyncio.gather() function has an issue: in case of error in one of coroutines the rest coroutines are left running detached. This might cause hard to detect problems. On the contrary, the async_plus.try_gather() insures all tasks are cancelled on error:

result1, result2 = await async_plus.try_gather(
    coroutine_func1(...),
    coroutine_func2(...),
)

Fire-and-forget task

With create_task() it’s your responsibility to retrieve exception. Usually this causes that exception is not seen until process finishes or even is not seen at all if process is killed. With launch_watched() excpetion is logged immediately when it’s raised.

async_plus.launch_watched(your_coroutine_func(...))

Structuring groups of tasks

async with async_plus.task_scope() as scope:
    scope.launch(coroutine_func1(...))
    scope.launch(coroutine_func2(...))
    await scope.wait()

By default, wait() call returns when all tasks finish or first exception occurs. In all cases all unfinished tasks are cancelled at the end of async with block.

Increase delay between attempts in supervisor

retry_delayer = async_plus.RetryDelayer([0, 10, 60], random_shift=1)
while True:
    try:
        await run_service_x()
    # In Python <3.8 it inherits from Exception
    except asyncio.CancelledError:
        raise
    except Exception:
        logger.exception('Error in service X:')
        await retry_delayer.sleep()

Log long waits

Does your program hang and you don’t know what it waits for? Wrap suspicious coroutines with impatient() to see bottlenecks:

await async_plus.impatient(asyncio.sleep(10), log_after=5)

Or just log time it took:

await async_plus.impatient(asyncio.sleep(10), log_completion='always')

Change log

See CHANGELOG.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

async_plus-0.2.0.tar.gz (8.9 kB view details)

Uploaded Source

Built Distribution

async_plus-0.2.0-py3-none-any.whl (6.0 kB view details)

Uploaded Python 3

File details

Details for the file async_plus-0.2.0.tar.gz.

File metadata

  • Download URL: async_plus-0.2.0.tar.gz
  • Upload date:
  • Size: 8.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.49.0 CPython/3.8.3

File hashes

Hashes for async_plus-0.2.0.tar.gz
Algorithm Hash digest
SHA256 a19a4476cbce609becd32a321de9daa0a7f93faea7625ec35e184ba22b50427c
MD5 b03d4c30ba09c10c0197e4344de95eae
BLAKE2b-256 c9ad893dd38a241be17266c7bf263bcd2ea80df8902a63c20e05f5ae3821a852

See more details on using hashes here.

File details

Details for the file async_plus-0.2.0-py3-none-any.whl.

File metadata

  • Download URL: async_plus-0.2.0-py3-none-any.whl
  • Upload date:
  • Size: 6.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.49.0 CPython/3.8.3

File hashes

Hashes for async_plus-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 b27d9f2ebf06888a01f274623be2dfe44f644cd9a6811bc9a8371e038130e8b8
MD5 ce28a9f75a15e61f34c68f15a9aa797c
BLAKE2b-256 645fd6ab71cfd387c5491c36b2258fc6db46cfc54eec3986be0148d6c6d160e0

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page