Skip to main content

Exports task execution metrics in Prometheus format

Project description

Exports task execution metrics in Prometheus format: how many tasks were started and have completed successfully or with failure, and how many tasks are still in the queues (supported only for broker redis).

Inspired by https://gitlab.com/kalibrr/celery-prometheus

Usage

Start HTTP service

Start the HTTP server like this:

$ bin/celery prometheus --host=127.0.0.1 --port=9691

Configure Prometheus

scrape_configs:
  - job_name: 'celery'
    static_configs:
      - targets: ['localhost:9691']

We export the following metrics:

  • celery_tasks_total{state="started|succeeded|failed|retried|retries-exceeded", queue="..."}, counter

  • celery_task_queuetime_seconds{queue}, histogram (only if task_send_sent_event is enabled in Celery)

  • celery_task_runtime_seconds{queue}, histogram

If you pass --queuelength-interval=x then every x seconds the queue lengths will be checked (NOTE: this only works with redis as the broker), resulting in this additional metric:

  • celery_queue_length{queue="..."}, gauge

Run tests

Using tox and py.test. Maybe install tox (e.g. via pip install tox) and then simply run tox.

celery_redis_prometheus changes

1.5.1 (2023-10-27)

  • Fix brown-bag release

1.5.0 (2023-10-27)

  • Be explicit about unregistering default collectors

1.4.0 (2022-10-28)

  • Add queue label to task_queuetime and task_runtime metrics

1.3.0 (2022-09-14)

  • Add queue label to celery_tasks_total metric

  • Add separate state retries-exceeded (previously used failed)

1.2.0 (2022-06-23)

  • Update to celery-5.x

1.1.1 (2020-03-20)

  • Fix py2 class syntax issues

1.1.0 (2020-03-20)

  • Make py2 compatible.

1.0.0 (2020-03-20)

  • Initial release

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

celery_redis_prometheus-1.5.1.tar.gz (6.6 kB view details)

Uploaded Source

File details

Details for the file celery_redis_prometheus-1.5.1.tar.gz.

File metadata

File hashes

Hashes for celery_redis_prometheus-1.5.1.tar.gz
Algorithm Hash digest
SHA256 54d9594facc82fb369ecb38f141f4aaf70615755ad6267b36d8817d33e3424db
MD5 a35c84acd2ae733681c4b99c0c47637c
BLAKE2b-256 25a778a98b9e1aafa11488f5a25942d2ba4ae05f465471129e0ddf0c687f8082

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page