Skip to main content

ZeroMQ integration with asyncio.

Project description

asyncio (PEP 3156) support for ZeroMQ.

https://travis-ci.com/aio-libs/aiozmq.svg?branch=master

The difference between aiozmq and vanilla pyzmq (zmq.asyncio).

zmq.asyncio works only by replacement event loop with custom one. This approach works but have two disadvantages:

  1. zmq.asyncio.ZMQEventLoop cannot be combined with other loop implementations (most notable is ultra fast uvloop).

  2. It uses internal ZMQ Poller which has fast ZMQ Sockets support but doesn’t intended to work fast with many (thousands) regular TCP sockets.

    In practice it means that zmq.asyncio is not recommended to be used with web servers like aiohttp.

    See also https://github.com/zeromq/pyzmq/issues/894

Documentation

See http://aiozmq.readthedocs.org

Simple high-level client-server RPC example:

import asyncio
import aiozmq.rpc


class ServerHandler(aiozmq.rpc.AttrHandler):

    @aiozmq.rpc.method
    def remote_func(self, a:int, b:int) -> int:
        return a + b


@asyncio.coroutine
def go():
    server = yield from aiozmq.rpc.serve_rpc(
        ServerHandler(), bind='tcp://127.0.0.1:5555')
    client = yield from aiozmq.rpc.connect_rpc(
        connect='tcp://127.0.0.1:5555')

    ret = yield from client.call.remote_func(1, 2)
    assert 3 == ret

    server.close()
    client.close()

asyncio.get_event_loop().run_until_complete(go())

Low-level request-reply example:

import asyncio
import aiozmq
import zmq

@asyncio.coroutine
def go():
    router = yield from aiozmq.create_zmq_stream(
        zmq.ROUTER,
        bind='tcp://127.0.0.1:*')

    addr = list(router.transport.bindings())[0]
    dealer = yield from aiozmq.create_zmq_stream(
        zmq.DEALER,
        connect=addr)

    for i in range(10):
        msg = (b'data', b'ask', str(i).encode('utf-8'))
        dealer.write(msg)
        data = yield from router.read()
        router.write(data)
        answer = yield from dealer.read()
        print(answer)
    dealer.close()
    router.close()

asyncio.get_event_loop().run_until_complete(go())

Comparison to pyzmq

zmq.asyncio provides a asyncio compatible loop implementation.

But it’s based on zmq.Poller which doesn’t work well with massive non-zmq sockets usage.

E.g. if you build a web server for handling at least thousands of parallel web requests (1000-5000) pyzmq internal Poller will be slow.

aiozmq works with epoll natively, it doesn’t need custom loop implementation and cooperates pretty well with uvloop for example.

For details see https://github.com/zeromq/pyzmq/issues/894

Requirements

License

aiozmq is offered under the BSD license.

CHANGES

0.8.0 (2016-12-07)

  • Respect events_backlog parameter in zmq stream creation #86

0.7.1 (2015-09-20)

  • Fix monitoring events implementation

  • Make the library compatible with Python 3.5

0.7.0 (2015-07-31)

  • Implement monitoring ZMQ events #50

  • Do deeper lookup for inhereted classes #54

  • Relax endpont check #56

  • Implement monitoring events for stream api #52

0.6.1 (2015-05-19)

  • Dynamically get list of pyzmq socket types

0.6.0 (2015-02-14)

  • Process asyncio specific exceptions as builtins.

  • Add repr(exception) to rpc server call logs if any

  • Add transport.get_write_buffer_limits() method

  • Add __repr__ to transport

  • Add zmq_type to tr.get_extra_info()

  • Add zmq streams

0.5.2 (2014-10-09)

  • Poll events after sending zmq message for eventless transport

0.5.1 (2014-09-27)

  • Fix loopless transport implementation.

0.5.0 (2014-08-23)

  • Support zmq devices in aiozmq.rpc.serve_rpc()

  • Add loopless 0MQ transport

0.4.1 (2014-07-03)

  • Add exclude_log_exceptions parameter to rpc servers.

0.4.0 (2014-05-28)

  • Implement pause_reading/resume_reading methods in ZmqTransport.

0.3.0 (2014-05-17)

  • Add limited support for Windows.

  • Fix unstable test execution, change ZmqEventLoop to use global shared zmq.Context by default.

  • Process cancellation on rpc servers and clients.

0.2.0 (2014-04-18)

  • msg in msg_received now is a list, not tuple

  • Allow to send empty msg by trsansport.write()

  • Add benchmarks

  • Derive ServiceClosedError from aiozmq.rpc.Error, not Exception

  • Implement logging from remote calls at server side (log_exceptions parameter).

  • Optimize byte counting in ZmqTransport.

0.1.3 (2014-04-10)

  • Function default values are not passed to an annotaion. Add check for libzmq version (should be >= 3.0)

0.1.2 (2014-04-01)

  • Function default values are not passed to an annotaion.

0.1.1 (2014-03-31)

  • Rename plural module names to single ones.

0.1.0 (2014-03-30)

  • Implement ZmqEventLoop with create_zmq_connection method which operates on zmq transport and protocol.

  • Implement ZmqEventLoopPolicy.

  • Introduce ZmqTransport and ZmqProtocol.

  • Implement zmq.rpc with RPC, PUSHPULL and PUBSUB protocols.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

aiozmq-0.9.0a0.tar.gz (29.8 kB view details)

Uploaded Source

Built Distribution

aiozmq-0.9.0a0-py3-none-any.whl (36.1 kB view details)

Uploaded Python 3

File details

Details for the file aiozmq-0.9.0a0.tar.gz.

File metadata

  • Download URL: aiozmq-0.9.0a0.tar.gz
  • Upload date:
  • Size: 29.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/45.1.0 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.6.7

File hashes

Hashes for aiozmq-0.9.0a0.tar.gz
Algorithm Hash digest
SHA256 f571ac2a4364afc1c3a6fe1f7081f82ece563a92947bdc508e8760c3b098a637
MD5 c9fcca96d7b34f98fad97351207744e9
BLAKE2b-256 94a609c3a5d624e1fbde13c1974490bb13f93cb0e5cff37d243ce687d7a1c91e

See more details on using hashes here.

Provenance

File details

Details for the file aiozmq-0.9.0a0-py3-none-any.whl.

File metadata

  • Download URL: aiozmq-0.9.0a0-py3-none-any.whl
  • Upload date:
  • Size: 36.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/45.1.0 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.6.7

File hashes

Hashes for aiozmq-0.9.0a0-py3-none-any.whl
Algorithm Hash digest
SHA256 feb18d2e8f1506454597c247e79026cc9f054f43409b861eda396c30620c4c75
MD5 964fae708cd993584da3a78304c8cfa9
BLAKE2b-256 01ca2c2e9b82955f75b359bcb7678a14e7ff53632a0b4ae445ea22dc1dcd7828

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page