Skip to main content

A Django middleware that logs tracebacks of slow requests to Sentry.

Project description

Overview

Dogslow is a Django watchdog middleware that logs tracebacks of slow requests. Dogslow-sentry requires Python 3.7+, Django 2.2+.

It started as an internal project inside Bitbucket to help trace operational problems.

In 2021, the dogslow-sentry fork was created to add Sentry-specific information to reports, like full stack trace, request information, fingerprint for issue grouping, breadcrumbs, etc.

Installation

Install dogslow-sentry:

$ pip install dogslow-sentry

Then add dogslow_sentry.WatchdogMiddleware to your Django settings file:

MIDDLEWARE = [
    'dogslow_sentry.WatchdogMiddleware',
    ...
]

For best results, make it one of the first middlewares that is run.

Configuration

Naturally, dogslow-sentry expects a working Sentry configuration for Django.

You can use the following configuration in your settings.py file to tune the watchdog:

# Watchdog is enabled by default, to temporarily disable, set to False:
DOGSLOW = True

# Log requests taking longer than 25 seconds:
DOGSLOW_TIMER = 25

# Enable logging to Sentry
DOGSLOW_SENTRY = True

# Also log slow request tracebacks to Python logger
DOGSLOW_LOGGER = 'dogslow_sentry'
DOGSLOW_LOG_LEVEL = 'WARNING'

# Tuple of url pattern names that should not be monitored:
# (defaults to none -- everything monitored)
DOGSLOW_IGNORE_URLS = ('some_view', 'other_view')

Usage

Every incoming HTTP request gets a 25 second timeout in the watchdog. If a request does not return within that time, the watchdog activates and takes a peek at the request thread’s stack and writes the backtrace (including all local stack variables – Django style) to a log file.

Note that dogslow only takes a peek at the thread’s stack. It does not interrupt the request, or influence it in any other way. Using dogslow is therefore safe to use in production.

Caveats

Dogslow uses multithreading. It has a single background thread that handles the watchdog timeouts and takes the tracebacks, so that the original request threads are not interrupted. This has some consequences.

Multithreading and the GIL

In CPython, the GIL (Global Interpreter Lock) prevents multiple threads from executing Python code simultaneously. Only when a thread explicitly releases its lock on the GIL, can a second thread run.

Releasing the GIL is done automatically whenever a Python program makes blocking calls outside of the interpreter, for example when doing IO.

For dogslow this means that it can only reliably intercept requests that are slow because they are doing IO, calling sleep or busy waiting to acquire locks themselves.

In most cases this is fine. An important cause of slow Django requests is an expensive database query. Since this is IO, dogslow can intercept those fine. A scenario where CPython’s GIL is problematic is when the request’s thread hits an infinite loop in Python code (or legitimate Python that is extremely expensive and takes a long time to execute), never releasing the GIL. Even though dogslow’s watchdog timer thread does become runnable, it cannot log the stack.

Co-routines and Greenlets

Dogslow is intended for use in a synchronous worker configuration. A webserver that uses dedicated threads (or single-threaded, dedicated worker processes) to serve requests. Django’s built-in wsgi server does this, as does Gunicorn in its default sync-worker mode.

When running with a “co-routines framework” where multiple requests are served concurrently by one thread, backtraces might become nonsensical.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dogslow-sentry-2.0.0b1.tar.gz (19.2 kB view details)

Uploaded Source

Built Distribution

dogslow_sentry-2.0.0b1-py3-none-any.whl (18.4 kB view details)

Uploaded Python 3

File details

Details for the file dogslow-sentry-2.0.0b1.tar.gz.

File metadata

  • Download URL: dogslow-sentry-2.0.0b1.tar.gz
  • Upload date:
  • Size: 19.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.6.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.9.6

File hashes

Hashes for dogslow-sentry-2.0.0b1.tar.gz
Algorithm Hash digest
SHA256 0aeee6fda56ceb8db23fadbfa149d08090a6c1c4ad8662c197a8aaf4a9beec80
MD5 6e88f9aaa237c56d8cccc43c318e7bcb
BLAKE2b-256 8d4708780ee2982b5b84a24be1526c9754777ddef1bb41c6a997f2917f361de3

See more details on using hashes here.

File details

Details for the file dogslow_sentry-2.0.0b1-py3-none-any.whl.

File metadata

  • Download URL: dogslow_sentry-2.0.0b1-py3-none-any.whl
  • Upload date:
  • Size: 18.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.6.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.9.6

File hashes

Hashes for dogslow_sentry-2.0.0b1-py3-none-any.whl
Algorithm Hash digest
SHA256 e8ad99f550081b72949a4173a27436fec53a6346b3191bdf92c944f83e47cccb
MD5 3a60d4ca6a24a21f5134b1dcca167331
BLAKE2b-256 d7359f20b4557667720896661a53dd7da647849b2d7e4125c6e17264ded2f661

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page