Skip to main content

evbus app-merge components for kafka queues

Project description

Made with pop, a Python implementation of Plugin Oriented Programming Made with Python

This project contains app-merge components for pop-evbus .

Getting Started

Prerequisites

  • Python 3.7+

  • git (if installing from source, or contributing to the project)

Installation

If wanting to use evbus-kafka, you can do so by either installing from PyPI or from source.

Install from PyPI

pip install evbus-kafka

Install from source

# clone repo
git clone git@gitlab.com:vmware/idem/evbus-kafka.git
cd evbus-kafka

# Setup venv
python3 -m venv .venv
source .venv/bin/activate
pip install -e .

Usage

Configure pop-evbus for your app.

The connection portion of the kafka profile accepts any parameters that can be used in the AIOKafkaProducer class

credentials.yml

kafka:
  default:
    connection:
      bootstrap_servers:
        - localhost:9092
    topics:
      - topic1
      - topic2

Encrypt the credentials file and export the ACCT environment variables

$ pip install acct
$ export ACCT_KEY=$(acct encrypt credentials.yml)
$ export ACCT_FILE="$PWD/credentials.yml.fernet"

Now when you put a message on the evbus queue, it will be propagated to your configured kafka implementation.

async def my_func(hub):
    await hub.evbus.broker.put(body={"message": "event content"}, profile="default")

Testing

A kafka server is necessary for running the tests. You can use your own kafka server, or you can use docker . Start a local kafka server:

docker run -p 2181:2181 -p 443:9092 -p 9092:9092 \                                      14s   idem-core 
  --env ADVERTISED_LISTENERS=PLAINTEXT://localhost:443,INTERNAL://localhost:9093 \
  --env LISTENERS=PLAINTEXT://0.0.0.0:9092,INTERNAL://0.0.0.0:9093 \
  --env SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,INTERNAL:PLAINTEXT \
  --env INTER_BROKER=INTERNAL \
  krisgeus/docker-kafka

Configure credentials for testing with a local rabbitmq server:

# credentials.yml
kafka:
  test_development_evbus_kafka:
    connection:
      bootstrap_servers: localhost:9092

Encrypt the credentials file and export the ACCT environment variables

$ pip install acct
$ export ACCT_KEY=$(acct encrypt credentials.yml)
$ export ACCT_FILE="$PWD/credentials.yml.fernet"

Install testing requirements

$ pip install -r requirements/test.in

Run the tests with pytest: .. code-block:: bash

$ pytest tests

Roadmap

Reference the open issues for a list of proposed features (and known issues).

Acknowledgements

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

evbus-kafka-4.0.0.tar.gz (9.7 kB view details)

Uploaded Source

Built Distribution

evbus_kafka-4.0.0-py3-none-any.whl (9.6 kB view details)

Uploaded Python 3

File details

Details for the file evbus-kafka-4.0.0.tar.gz.

File metadata

  • Download URL: evbus-kafka-4.0.0.tar.gz
  • Upload date:
  • Size: 9.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/35.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.9 tqdm/4.64.0 importlib-metadata/4.11.3 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.12

File hashes

Hashes for evbus-kafka-4.0.0.tar.gz
Algorithm Hash digest
SHA256 7f7996621cd83e51a86fccd2afe1c3602b884c6f266cdeb5830c74afb50f27a5
MD5 6520093b2534a76845e223b9f10ba83b
BLAKE2b-256 a73c49f6f44ddc9a887e73a8b13ee514cd74e09edef65842d97f4c01687726ea

See more details on using hashes here.

File details

Details for the file evbus_kafka-4.0.0-py3-none-any.whl.

File metadata

  • Download URL: evbus_kafka-4.0.0-py3-none-any.whl
  • Upload date:
  • Size: 9.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/35.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.9 tqdm/4.64.0 importlib-metadata/4.11.3 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.12

File hashes

Hashes for evbus_kafka-4.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 1e79cec9c8afc37a834b02bef69dbe3b4c75c2d8205d1b12c799db58c20a5fcd
MD5 a6fc452aa5f8e5be0bb3f54db18233b3
BLAKE2b-256 eea1302c077aaa968f23a521dcfa9c80043e7a54f89d07c56522d094c223a242

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page