Round-robin multidb router for Django.
Project description
``multidb`` provides two Django database routers useful in primary-replica database
deployments.
ReplicaRouter
-----------------
With ``multidb.ReplicaRouter`` all read queries will go to a replica
database; all inserts, updates, and deletes will go to the ``default``
database.
First, define ``REPLICA_DATABASES`` in your settings. It should be a list of
database aliases that can be found in ``DATABASES``::
DATABASES = {
'default': {...},
'shadow-1': {...},
'shadow-2': {...},
}
REPLICA_DATABASES = ['shadow-1', 'shadow-2']
Then put ``multidb.ReplicaRouter`` into DATABASE_ROUTERS::
DATABASE_ROUTERS = ('multidb.ReplicaRouter',)
The replica databases will be chosen in round-robin fashion.
If you want to get a connection to a replica in your app, use
``multidb.get_replica``::
from django.db import connections
import multidb
connection = connections[multidb.get_replica()]
PinningReplicaRouter
------------------------
In some applications, the lag between the primary database receiving a write and its
replication to the replicas is enough to cause inconsistency for the end user.
For example, imagine a scenario with 1 second of replication lag. If a user
makes a forum post (to the primary) and then is redirected to a fully-rendered
view of it (from a replica) 500ms later, the view will fail. If this is a problem
in your application, consider using ``multidb.PinningReplicaRouter``. This
router works in combination with ``multidb.middleware.PinningRouterMiddleware``
to assure that, after writing to the ``default`` database, future reads from
the same user agent are directed to the ``default`` database for a configurable
length of time.
Caveats
=======
``PinningRouterMiddleware`` identifies database writes primarily by request
type, assuming that requests with HTTP methods that are not ``GET``, ``TRACE``,
``HEAD``, or ``OPTIONS`` are writes. You can indicate that any view writes to
the database by using the ``multidb.db_write`` decorator. This will cause the
same result as if the request were, e.g., a ``POST``.
You can also manually set ``response._db_write = True`` to indicate that a
write occurred. This will not result in using the ``default`` database in this
request, but only in the next request.
Configuration
=============
To use ``PinningReplicaRouter``, put it into ``DATABASE_ROUTERS`` in your
settings::
DATABASE_ROUTERS = ('multidb.PinningReplicaRouter',)
Then, install the middleware. It must be listed before any other middleware
which performs database writes::
MIDDLEWARE_CLASSES = (
'multidb.middleware.PinningRouterMiddleware',
...more middleware here...
)
``PinningRouterMiddleware`` attaches a cookie to any user agent who has just
written. The cookie should be set to expire at a time longer than your
replication lag. By default, its value is a conservative 15 seconds, but it can
be adjusted like so::
MULTIDB_PINNING_SECONDS = 5
If you need to change the name of the cookie, use the ``MULTIDB_PINNING_COOKIE``
setting::
MULTIDB_PINNING_COOKIE = 'multidb_pin_writes'
``use_primary_db``
==============
``multidb.pinning.use_primary_db`` is both a context manager and a decorator for
wrapping code to use the primary database. You can use it as a context manager::
from multidb.pinning import use_primary_db
with use_primary_db:
touch_the_database()
touch_another_database()
or as a decorator::
from multidb.pinning import use_primary_db
@use_primary_db
def func(*args, **kw):
"""Touches the primary database."""
Running the Tests
-----------------
To run the tests, you'll need to install the development requirements::
pip install -r requirements.txt
./run.sh test
Alternatively, you can run the tests with several versions of Django
and Python using tox:
$ pip install tox
$ tox
deployments.
ReplicaRouter
-----------------
With ``multidb.ReplicaRouter`` all read queries will go to a replica
database; all inserts, updates, and deletes will go to the ``default``
database.
First, define ``REPLICA_DATABASES`` in your settings. It should be a list of
database aliases that can be found in ``DATABASES``::
DATABASES = {
'default': {...},
'shadow-1': {...},
'shadow-2': {...},
}
REPLICA_DATABASES = ['shadow-1', 'shadow-2']
Then put ``multidb.ReplicaRouter`` into DATABASE_ROUTERS::
DATABASE_ROUTERS = ('multidb.ReplicaRouter',)
The replica databases will be chosen in round-robin fashion.
If you want to get a connection to a replica in your app, use
``multidb.get_replica``::
from django.db import connections
import multidb
connection = connections[multidb.get_replica()]
PinningReplicaRouter
------------------------
In some applications, the lag between the primary database receiving a write and its
replication to the replicas is enough to cause inconsistency for the end user.
For example, imagine a scenario with 1 second of replication lag. If a user
makes a forum post (to the primary) and then is redirected to a fully-rendered
view of it (from a replica) 500ms later, the view will fail. If this is a problem
in your application, consider using ``multidb.PinningReplicaRouter``. This
router works in combination with ``multidb.middleware.PinningRouterMiddleware``
to assure that, after writing to the ``default`` database, future reads from
the same user agent are directed to the ``default`` database for a configurable
length of time.
Caveats
=======
``PinningRouterMiddleware`` identifies database writes primarily by request
type, assuming that requests with HTTP methods that are not ``GET``, ``TRACE``,
``HEAD``, or ``OPTIONS`` are writes. You can indicate that any view writes to
the database by using the ``multidb.db_write`` decorator. This will cause the
same result as if the request were, e.g., a ``POST``.
You can also manually set ``response._db_write = True`` to indicate that a
write occurred. This will not result in using the ``default`` database in this
request, but only in the next request.
Configuration
=============
To use ``PinningReplicaRouter``, put it into ``DATABASE_ROUTERS`` in your
settings::
DATABASE_ROUTERS = ('multidb.PinningReplicaRouter',)
Then, install the middleware. It must be listed before any other middleware
which performs database writes::
MIDDLEWARE_CLASSES = (
'multidb.middleware.PinningRouterMiddleware',
...more middleware here...
)
``PinningRouterMiddleware`` attaches a cookie to any user agent who has just
written. The cookie should be set to expire at a time longer than your
replication lag. By default, its value is a conservative 15 seconds, but it can
be adjusted like so::
MULTIDB_PINNING_SECONDS = 5
If you need to change the name of the cookie, use the ``MULTIDB_PINNING_COOKIE``
setting::
MULTIDB_PINNING_COOKIE = 'multidb_pin_writes'
``use_primary_db``
==============
``multidb.pinning.use_primary_db`` is both a context manager and a decorator for
wrapping code to use the primary database. You can use it as a context manager::
from multidb.pinning import use_primary_db
with use_primary_db:
touch_the_database()
touch_another_database()
or as a decorator::
from multidb.pinning import use_primary_db
@use_primary_db
def func(*args, **kw):
"""Touches the primary database."""
Running the Tests
-----------------
To run the tests, you'll need to install the development requirements::
pip install -r requirements.txt
./run.sh test
Alternatively, you can run the tests with several versions of Django
and Python using tox:
$ pip install tox
$ tox
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file django-multidb-router-0.9.tar.gz
.
File metadata
- Download URL: django-multidb-router-0.9.tar.gz
- Upload date:
- Size: 8.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.8.0 tqdm/4.29.0 CPython/3.6.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | d8096eff3a42743a795bb943260758d169bc78de61e12eeb0893d6dbfbae72d0 |
|
MD5 | ea3651565d9f30cd96b318a92b414b01 |
|
BLAKE2b-256 | 68195f48a52eb03b6ee630ab318386c94597bb09956dc930520ff4ca0bf7f764 |
Provenance
File details
Details for the file django_multidb_router-0.9-py3-none-any.whl
.
File metadata
- Download URL: django_multidb_router-0.9-py3-none-any.whl
- Upload date:
- Size: 9.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.8.0 tqdm/4.29.0 CPython/3.6.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 38a006455fa1141a332029791863461c0b1f287b8c479b411b4d9f728e86e97a |
|
MD5 | 5fe5bc9f3af0e7ced2d38fae94631bee |
|
BLAKE2b-256 | 4ecf5e6a92d88e54ac532dfee9b69d260678ca8af5bbac5164e3bafdb58ba4d8 |