Skip to main content

Python client for managing Kafka connectors..

Project description

#####################
kafka-connect-manager
#####################

|Build| |Docker|

A Python client to configure and create connectors using the `Kafka Connect API <https://docs.confluent.io/current/connect/references/restapi.html>`_.

Overview
========

Kafka-connect-manager helps to configure and create Kafka connectors. It can select Kafka topics using a regular expression and exclude topics added to a exclusion list. The auto-update option dynamically updates the connector if topics are created or deleted in Kafka.

Kafka-connect-manager can validate the connector configuration and return the validation output in case of errors.

Kafka-connect-manager provides a command line interface (CLI) to interact with the `Kafka Connect API <https://docs.confluent.io/current/connect/references/restapi.html>`_. Using the CLI you can manage existing connectors or create a new one.

To deploy a supported connector, you can use the `kafka-connect-manager Helm chart <https://lsst-sqre.github.io/charts/>`_.

Supported connectors
--------------------

* `Lenses InfluxDB Sink <https://docs.lenses.io/connectors/sink/influx.html>`_
* `Amazon S3 Sink <https://docs.confluent.io/current/connect/kafka-connect-s3>`_

See `the docs <https://kafka-connect-manager.lsst.io>`_ for more information.


.. |Build| image:: https://github.com/lsst-sqre/kafka-connect-manager/workflows/CI/badge.svg
:alt: GitHub Actions
:scale: 100%
:target: https://github.com/lsst-sqre/kafka-connect-manager/actions

.. |Docker| image:: https://img.shields.io/docker/v/lsstsqre/kafkaconnect?sort=date
:alt: Docker Hub repository
:scale: 100%
:target: https://hub.docker.com/repository/docker/lsstsqre/kafkaconnect

##########
Change log
##########

0.8.0 (2020-08-05)
==================

* Use data classes for the application and connector configuration.
* Plugin like organization, to support new connectors add a cli and a config file.
* Add support to the Amazon S3 Sink connector

0.7.2 (2020-03-31)
==================

* Add support to the InfluxDB Sink Connector.
* Add --timestamp option to select the timestamp field to use in the InfluxDB Sink connector.
* Fix Header Converter Class configuration setting.
* Fix ``tasks.max`` configuration setting name.
* Add connector ``name`` configuration setting to support multiple connectors of the same class.
* Handle empty list of topics properly.


Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

kafkaconnect-0.8.0.tar.gz (51.9 kB view details)

Uploaded Source

Built Distribution

kafkaconnect-0.8.0-py3-none-any.whl (16.9 kB view details)

Uploaded Python 3

File details

Details for the file kafkaconnect-0.8.0.tar.gz.

File metadata

  • Download URL: kafkaconnect-0.8.0.tar.gz
  • Upload date:
  • Size: 51.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/49.3.1 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for kafkaconnect-0.8.0.tar.gz
Algorithm Hash digest
SHA256 294350304f99784da9c447b06264e4d29e5aa76e709c8e4a83e357e54fa5d80b
MD5 dcea585d14e1988d5ded91eda23eade3
BLAKE2b-256 cf0b621717f62771466511b0c6a2d11c4eff67a04fd532ebc59c031169edf800

See more details on using hashes here.

File details

Details for the file kafkaconnect-0.8.0-py3-none-any.whl.

File metadata

  • Download URL: kafkaconnect-0.8.0-py3-none-any.whl
  • Upload date:
  • Size: 16.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/49.3.1 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for kafkaconnect-0.8.0-py3-none-any.whl
Algorithm Hash digest
SHA256 e11cd1988557aa3e3861bd87dba9b91b3bc3c82cd4581156283ad4dcfa3756a2
MD5 ad98f80fda73f8884f1ef3b618e50e33
BLAKE2b-256 9cf2e239928fe98548ddea6592b925286b06ef6c8db9cbd6b46484eca0071ab3

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page