Skip to main content

Arroyo is a Python library for working with streaming data.

Project description

Arroyo

Arroyo is a library to build streaming applications that consume from and produce to Kafka.

Arroyo consists of three components:

  • Consumer and producer backends
    • The Kafka backend is a wrapper around the librdkafka client, and attempts to simplify rebalancing and offset management even further
    • There is also an in memory and a file based consumer and producer implementation that can be used for testing
  • A strategy interface
    • Arroyo includes a number of pre-built strategies such as RunTask, Filter, Reduce, CommitOffsets and more.
    • Users can write their own strategies, though in most cases this should not be needed as the library aims to provide generic, reusable strategies that cover most stream processing use cases
    • Strategies can be chained together to form complex message processing pipelines.
  • A streaming engine which manages the relationship between the consumer and strategies
    • The StreamProcessor controls progress by the consumer and schedules work for execution by the strategies.

All documentation is in the docs directory. It is hosted at https://getsentry.github.io/arroyo/ and can be built locally by running make docs

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sentry-arroyo-2.10.5.tar.gz (65.7 kB view details)

Uploaded Source

Built Distribution

sentry_arroyo-2.10.5-py3-none-any.whl (91.3 kB view details)

Uploaded Python 3

File details

Details for the file sentry-arroyo-2.10.5.tar.gz.

File metadata

  • Download URL: sentry-arroyo-2.10.5.tar.gz
  • Upload date:
  • Size: 65.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.4.2 requests/2.25.1 setuptools/52.0.0 requests-toolbelt/0.9.1 tqdm/4.57.0 CPython/3.9.2

File hashes

Hashes for sentry-arroyo-2.10.5.tar.gz
Algorithm Hash digest
SHA256 dd7bebb57e647e62753fcd9c9dd457f3925463577fb4877ef2c7d3ab1c422b7f
MD5 c86063e503e6dc900bd52262386b09bb
BLAKE2b-256 932ad4e0f95ffbcfcda53314671000850517eaf0d389cd6f84dca3095cde294b

See more details on using hashes here.

File details

Details for the file sentry_arroyo-2.10.5-py3-none-any.whl.

File metadata

  • Download URL: sentry_arroyo-2.10.5-py3-none-any.whl
  • Upload date:
  • Size: 91.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.4.2 requests/2.25.1 setuptools/52.0.0 requests-toolbelt/0.9.1 tqdm/4.57.0 CPython/3.9.2

File hashes

Hashes for sentry_arroyo-2.10.5-py3-none-any.whl
Algorithm Hash digest
SHA256 eb5dc07390b41c016e948a81acffaf10a9a7528a71843854084680adf06e7f46
MD5 f3f7e9584e40c3e7a18931dbd5db67fb
BLAKE2b-256 58d12688354e4ca88ae4339c19b84eaeee6cad64de0db0c4f1f3da24dba5ee9f

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page