Detecting the n+1 queries problem in Python
Project description
nplusone is a library for detecting the n+1 queries problem in Python ORMs, including SQLAlchemy, Peewee, and the Django ORM.
The Problem
Many object-relational mapping (ORM) libraries default to lazy loading for relationships. This pattern can be efficient when related rows are rarely accessed, but quickly becomes inefficient as relationships are accessed more frequently. In these cases, loading related rows eagerly using a JOIN can be vastly more performant. Unfortunately, understanding when to use lazy versus eager loading can be challenging: you might not notice the problem until your app has slowed to a crawl.
nplusone is an ORM profiling tool to help diagnose and improve poor performance caused by inappropriate lazy loading. nplusone monitors applications using Django or SQLAlchemy and sends warnings when potentially expensive lazy loads are emitted. It can identify the offending relationship attribute and specific lines of code behind the problem, and recommend fixes for better performance.
nplusone also detects inappropriate eager loading for Flask-SQLAlchemy and the Django ORM, emitting a warning when related data are eagerly loaded but never accessed within the current request.
Installation
pip install -U nplusone
nplusone supports Python >= 2.7 or >= 3.3.
Usage
Note: nplusone should only be used for development and should not be deployed to production environments.
Django
Add nplusone to INSTALLED_APPS:
INSTALLED_APPS = ( ... 'nplusone.ext.django', )
Add NPlusOneMiddleware:
MIDDLEWARE_CLASSES = ( ... 'nplusone.ext.django.NPlusOneMiddleware', )
Optionally configure logging settings:
NPLUSONE_LOGGER = logging.getLogger('nplusone') NPLUSONE_LOG_LEVEL = logging.WARN
Configure logging handlers:
LOGGING = { 'version': 1, 'handlers': { 'console': { 'class': 'logging.StreamHandler', }, }, 'loggers': { 'nplusone': { 'handlers': ['console'], 'level': 'WARN', }, }, }
When your app loads data lazily, nplusone will emit a warning:
Potential n+1 query detected on `<model>.<field>`
Consider using select_related or prefetch_related in this case.
When your app eagerly loads related data without accessing it, nplusone will log a warning:
Potential unnecessary eager load detected on `<model>.<field>`
Flask-SQLAlchemy
Wrap application with NPlusOne:
from flask import Flask from nplusone.ext.flask_sqlalchemy import NPlusOne app = Flask(__name__) NPlusOne(app)
Optionally configure logging settings:
app = Flask(__name__) app.config['NPLUSONE_LOGGER'] = logging.getLogger('app.nplusone') app.config['NPLUSONE_LOG_LEVEL'] = logging.ERROR NPlusOne(app)
When your app loads data lazily, nplusone will emit a warning:
Potential n+1 query detected on `<model>.<field>`
Consider using subqueryload or joinedload in this case; see SQLAlchemy’s guide to relationship loading for complete documentation.
When your app eagerly loads related data without accessing it, nplusone will log a warning:
Potential unnecessary eager load detected on `<model>.<field>`
Ignoring Warnings
To suppress warnings thrown by intentional lazy loading, use the ignore context manager:
from nplusone.core import signals with signals.ignore(signals.lazy_load): # lazy-load rows # ...
License
MIT licensed. See the bundled LICENSE file for more details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file nplusone-0.2.0.tar.gz
.
File metadata
- Download URL: nplusone-0.2.0.tar.gz
- Upload date:
- Size: 8.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8c53d31d484dd6d1728f00eb8a17e92b29af9b5f3711e982f507c42352e4bd2e |
|
MD5 | bd5867b5cb7f0eabe6a12417a901236f |
|
BLAKE2b-256 | 4dda7f4810253e7726194d726ad2dc15404fcf6409a997b20fad75f5da02d8e2 |
File details
Details for the file nplusone-0.2.0-py2.py3-none-any.whl
.
File metadata
- Download URL: nplusone-0.2.0-py2.py3-none-any.whl
- Upload date:
- Size: 12.7 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9532cc700ca68100570768ac3190ac1e2a67427cad8f296165ca47dcfa481fc3 |
|
MD5 | 7dab7bec5a5a0ae0bf9c45598338c551 |
|
BLAKE2b-256 | 591fb2f5725af4affb8823e642fbb7823c48245c74a2c93fa1d1f2774072b168 |