Skip to main content

Postgres Broker for Dramatiq Task Queue

Project description

=============================================
dramatiq-pg -- Postgres Broker for Dramatiq
=============================================

dramatiq_ is a simple task queue implementation for Python3. dramatiq-pg
provides a Postgres-based implementation of a dramatiq broker.

**The project is not feature complete yet.**


Features
========

- Super simple deployment.
- Stores messages in a single table.
- All data are wrapped in a dedicated schema.
- Uses LISTEN/NOTIFY to keep worker sync. No polling.
- Reliable thanks to Postgres MVCC.
- Using plain psycopg2. No ORM.


Installation
============

- Install dramatiq-pg package from PyPI::

pip install dramatiq-pg

- Apply dramatiq_pg/schema.sql file in your database::

psql -f dramatiq_pg/schema.sql

- Before importing actors, define global broker with a connection pool::

import dramatiq
import dramatiq_pg
import psycopg2.pool

pool = psycopg2.pool.ThreadedConnectionPool(0, 4, conninfo)
dramatiq.set_broker(dramatiq_pg.PostgresBroker(pool=pool))

Now declare/import actors and manage worker just like any `dramatiq setup
<https://dramatiq.io/guide.html>`_.


Roadmap
=======

- Process missed notifyes while resuming worker.
- Functionnal tests.
- Result storage as JSONb.
- Delayed task.


.. _dramatiq:: https://dramatiq.io/

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dramatiq-pg-0.1.0.tar.gz (3.5 kB view details)

Uploaded Source

Built Distribution

dramatiq_pg-0.1.0-py3-none-any.whl (6.8 kB view details)

Uploaded Python 3

File details

Details for the file dramatiq-pg-0.1.0.tar.gz.

File metadata

  • Download URL: dramatiq-pg-0.1.0.tar.gz
  • Upload date:
  • Size: 3.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/0.12.10 CPython/3.7.1 Linux/4.19.0-2-amd64

File hashes

Hashes for dramatiq-pg-0.1.0.tar.gz
Algorithm Hash digest
SHA256 0ee2a909a10077592968a87d4f3edfd371d9fc10b0e05e4ea072f751f3f47f3a
MD5 ae5ada6b0e97f360f907efb279fd59bc
BLAKE2b-256 183afca6a4ddd0d8d1460d3132abdfb72097526d601e3999ef1e6e799576790d

See more details on using hashes here.

Provenance

File details

Details for the file dramatiq_pg-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: dramatiq_pg-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 6.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/0.12.10 CPython/3.7.1 Linux/4.19.0-2-amd64

File hashes

Hashes for dramatiq_pg-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 c3e33ffb41d10bd56852832ee9413a849fa398f39662d57be406fa556bfaff97
MD5 e9463486f2ef2e17f0f3ec45c3c6f5c2
BLAKE2b-256 ba073a312de31d7d2432d59b410226c774771bfff4e8366373c29b9bbcd120f6

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page