Callosum

An RPC Transport Library

It provides an asynchronous multi-channel order-preserving message and data streaming transport for upper RPC layers (e.g., Apache Thrift) by wrapping lower transport implementations (e.g., ZeroMQ).

It aims to follow the latest coding style and conventions in Python asyncio.

Corpus callosum is a bundle of neuron fibers that connects two cerebral hemispheres of a human brain.

Prerequisite

Python 3.8 or higher.

Features

• RPC
  • Native timeout and cancellation support
  • Explicit server-to-client error propagation including stringified tracebacks
  • Order preserving based on user-defined keys while keeping executions asynchronous
  • Concurrency limits based on aiojobs
• Streaming
  • Broadcast & shared pipelines
• Optional client authentication and encrypted communication
  • Currently supported for only ZeroMQ with its CURVE library
• Optional message compression using snappy
• Replacible and combinable lower/upper layers (ZeroMQ/Redis + JSON/msgpack/Thrift)

Planned features

• Managed streaming (with acks)
• Tunneling to bundle other channels and generic network traffic in a single connection
• Bidirectional RPC
• Chunked transfer of large messages

Installation

To install the core:

$ pip install -U pip setuptools
$ pip install callosum

You may add extra dependencies like:

$ pip install 'callosum[zeromq,redis,thrift,snappy]'

Examples

Please check out the examples directory.

Development

Use the editable installation of Python setuptools.

$ pip install -U pip setuptools
$ pip install -U -e '.[dev,build,test,zeromq,redis,thrift,snappy]'

Changes

v0.9.4 (2020-04-10)

Fixes

• Fix a race condition due to overlapping RPC message sequence IDs by separating server/client message sequence IDs (#12)

Miscellaneous

• Adopt towncrier for changelog management (#11)

v0.9.3 (2020-03-20)

• FIX: Mitigate bogus KeyError when cleaning up task futures in RPC servers that has caused event loop spinning.

v0.9.2 (2020-02-28)

• MAINTENANCE: Update dependencies and only specify the minimum versions since Callosum is a library.

v0.9.1 (2020-01-05)

• FIX: wrong typing of RPCMessage.body field

• IMPROVE: Add debug_rpc option to rpc.Peer for logging exceptions in RPC scheduler and user-defined handlers explicitly.

• Update dependencies and remove unused ones.

v0.9.0 (2019-12-06)

• First public release with a working RPC based on ZeroMQ DEALER/ROUTER sockets.

2018-05-02

• Started the project.