Python client for managing Kafka connectors..
Project description
#####################
kafka-connect-manager
#####################
|Build| |Docker|
A Python client to configure and create connectors using the `Kafka Connect API <https://docs.confluent.io/current/connect/references/restapi.html>`_.
Overview
========
Kafka-connect-manager helps to configure and create Kafka connectors. It can select Kafka topics using a regular expression and exclude topics added to a exclusion list. The auto-update option dynamically updates the connector if topics are created or deleted in Kafka.
Kafka-connect-manager can validate the connector configuration and return the validation output in case of errors.
Kafka-connect-manager provides a command line interface (CLI) to interact with the `Kafka Connect API <https://docs.confluent.io/current/connect/references/restapi.html>`_. Using the CLI you can manage existing connectors or create a new one.
To deploy a supported connector, you can use the `kafka-connect-manager Helm chart <https://lsst-sqre.github.io/charts/>`_.
Supported connectors
--------------------
* `Lenses InfluxDB Sink <https://docs.lenses.io/connectors/sink/influx.html>`_
* `Amazon S3 Sink <https://docs.confluent.io/current/connect/kafka-connect-s3>`_
See `the docs <https://kafka-connect-manager.lsst.io>`_ for more information.
.. |Build| image:: https://github.com/lsst-sqre/kafka-connect-manager/workflows/CI/badge.svg
:alt: GitHub Actions
:scale: 100%
:target: https://github.com/lsst-sqre/kafka-connect-manager/actions
.. |Docker| image:: https://img.shields.io/docker/v/lsstsqre/kafkaconnect?sort=date
:alt: Docker Hub repository
:scale: 100%
:target: https://hub.docker.com/repository/docker/lsstsqre/kafkaconnect
##########
Change log
##########
0.8.0 (2020-08-05)
==================
* Use data classes for the application and connector configuration.
* Plugin like organization, to support new connectors add a cli and a config file.
* Add support to the Amazon S3 Sink connector
0.7.2 (2020-03-31)
==================
* Add support to the InfluxDB Sink Connector.
* Add --timestamp option to select the timestamp field to use in the InfluxDB Sink connector.
* Fix Header Converter Class configuration setting.
* Fix ``tasks.max`` configuration setting name.
* Add connector ``name`` configuration setting to support multiple connectors of the same class.
* Handle empty list of topics properly.
kafka-connect-manager
#####################
|Build| |Docker|
A Python client to configure and create connectors using the `Kafka Connect API <https://docs.confluent.io/current/connect/references/restapi.html>`_.
Overview
========
Kafka-connect-manager helps to configure and create Kafka connectors. It can select Kafka topics using a regular expression and exclude topics added to a exclusion list. The auto-update option dynamically updates the connector if topics are created or deleted in Kafka.
Kafka-connect-manager can validate the connector configuration and return the validation output in case of errors.
Kafka-connect-manager provides a command line interface (CLI) to interact with the `Kafka Connect API <https://docs.confluent.io/current/connect/references/restapi.html>`_. Using the CLI you can manage existing connectors or create a new one.
To deploy a supported connector, you can use the `kafka-connect-manager Helm chart <https://lsst-sqre.github.io/charts/>`_.
Supported connectors
--------------------
* `Lenses InfluxDB Sink <https://docs.lenses.io/connectors/sink/influx.html>`_
* `Amazon S3 Sink <https://docs.confluent.io/current/connect/kafka-connect-s3>`_
See `the docs <https://kafka-connect-manager.lsst.io>`_ for more information.
.. |Build| image:: https://github.com/lsst-sqre/kafka-connect-manager/workflows/CI/badge.svg
:alt: GitHub Actions
:scale: 100%
:target: https://github.com/lsst-sqre/kafka-connect-manager/actions
.. |Docker| image:: https://img.shields.io/docker/v/lsstsqre/kafkaconnect?sort=date
:alt: Docker Hub repository
:scale: 100%
:target: https://hub.docker.com/repository/docker/lsstsqre/kafkaconnect
##########
Change log
##########
0.8.0 (2020-08-05)
==================
* Use data classes for the application and connector configuration.
* Plugin like organization, to support new connectors add a cli and a config file.
* Add support to the Amazon S3 Sink connector
0.7.2 (2020-03-31)
==================
* Add support to the InfluxDB Sink Connector.
* Add --timestamp option to select the timestamp field to use in the InfluxDB Sink connector.
* Fix Header Converter Class configuration setting.
* Fix ``tasks.max`` configuration setting name.
* Add connector ``name`` configuration setting to support multiple connectors of the same class.
* Handle empty list of topics properly.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
kafkaconnect-0.8.0.tar.gz
(51.9 kB
view details)
Built Distribution
File details
Details for the file kafkaconnect-0.8.0.tar.gz
.
File metadata
- Download URL: kafkaconnect-0.8.0.tar.gz
- Upload date:
- Size: 51.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/49.3.1 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 294350304f99784da9c447b06264e4d29e5aa76e709c8e4a83e357e54fa5d80b |
|
MD5 | dcea585d14e1988d5ded91eda23eade3 |
|
BLAKE2b-256 | cf0b621717f62771466511b0c6a2d11c4eff67a04fd532ebc59c031169edf800 |
File details
Details for the file kafkaconnect-0.8.0-py3-none-any.whl
.
File metadata
- Download URL: kafkaconnect-0.8.0-py3-none-any.whl
- Upload date:
- Size: 16.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/49.3.1 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e11cd1988557aa3e3861bd87dba9b91b3bc3c82cd4581156283ad4dcfa3756a2 |
|
MD5 | ad98f80fda73f8884f1ef3b618e50e33 |
|
BLAKE2b-256 | 9cf2e239928fe98548ddea6592b925286b06ef6c8db9cbd6b46484eca0071ab3 |