Skip to main content

Async-related stuff you miss in standard library

Project description

Async-related stuff you miss in standard library

https://upload.wikimedia.org/wikipedia/commons/thumb/f/fe/The_Hanging_by_Jacques_Callot.jpg/500px-The_Hanging_by_Jacques_Callot.jpg

“The horrors of hanging tasks when using asyncio” by Jacques Callot

Safely run coroutines concurrently

The asyncio.gather() function has an issue: in case of error in one of coroutines the rest coroutines are left running detached. This might cause hard to detect problems. On the contrary, the async_plus.try_gather() insures all tasks are cancelled on error:

result1, result2 = await async_plus.try_gather(
    coroutine_func1(...),
    coroutine_func2(...),
)

Fire-and-forget task

With create_task() it’s your responsibility to retrieve exception. Usually this causes that exception is not seen until process finishes or even is not seen at all if process is killed. With launch_watched() exception is logged immediately when it’s raised.

async_plus.launch_watched(your_coroutine_func(...))

Structuring groups of tasks

async with async_plus.task_scope() as scope:
    scope.launch(coroutine_func1(...))
    scope.launch(coroutine_func2(...))
    await scope.wait()

By default, wait() call returns when all tasks finish or first exception occurs. In all cases all unfinished tasks are cancelled at the end of async with block.

Increase delay between attempts in supervisor

retry_delayer = async_plus.RetryDelayer([0, 10, 60], random_shift=1)
while True:
    try:
        await run_service_x()
    # In Python <3.8 it inherits from Exception
    except asyncio.CancelledError:
        raise
    except Exception:
        logger.exception('Error in service X:')
        await retry_delayer.sleep()

Log long waits

Does your program hang and you don’t know what it’s waiting for? Wrap suspicious coroutines with impatient() to see bottlenecks:

await async_plus.impatient(asyncio.sleep(10), log_after=5)

Or just log the time it took:

await async_plus.impatient(asyncio.sleep(10), log_completion='always')

Change log

See CHANGELOG.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

async_plus-0.3.0.tar.gz (9.8 kB view details)

Uploaded Source

Built Distribution

async_plus-0.3.0-py3-none-any.whl (6.9 kB view details)

Uploaded Python 3

File details

Details for the file async_plus-0.3.0.tar.gz.

File metadata

  • Download URL: async_plus-0.3.0.tar.gz
  • Upload date:
  • Size: 9.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.57.0 CPython/3.8.5

File hashes

Hashes for async_plus-0.3.0.tar.gz
Algorithm Hash digest
SHA256 5910ac38609f2f0aaa0f668360f35aecf6849875e7064e3baa63e7d4e3a5b9c4
MD5 532cab35fd07b68e096096ac49fd626a
BLAKE2b-256 a67963cdc4ee0d87b7637f9a45b1b7454518a51e5c53879e850ee874d25813ef

See more details on using hashes here.

File details

Details for the file async_plus-0.3.0-py3-none-any.whl.

File metadata

  • Download URL: async_plus-0.3.0-py3-none-any.whl
  • Upload date:
  • Size: 6.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.57.0 CPython/3.8.5

File hashes

Hashes for async_plus-0.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 004938d3d922a7fe2dc9db95486b103b9ef2b64f5e887d8d9a60608a279dea40
MD5 88c13604c004b939d7eb0ef427f6e012
BLAKE2b-256 60818178c07df8869a4ac3cd66b946f540f18ca36c92108b9076a793af68361e

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page