django-ultracache 2.2

Overview

Cache views, template fragments and arbitrary Python code. Once cached we either avoid database queries and expensive computations, depending on the use case. In all cases affected caches are automatically expired when objects “red” or “blue” are modified, without us having to explicitly make the cache aware of “red” or “blue”.

View:

from ultracache.decorators import ultracache

# The decorator with no parameters automatically caches on the request path
# and a minimal
# set of hidden parameters.
@ultracache(300)
class MyView(TemplateView):
    template_name = "my_view.html"

    def get_context_data(self, **kwargs):
        context = super().get_context_data(**kwargs)

        # The URL that was called is eg. http://mysite.com/myview/blue.
        # This view remains cached until "blue" is modified by any other
        # code.
        context["color"] = Color.objects.get(slug=kwargs["color_slug"])

        # Retrieve "red". Even though we never use red further in this code
        # the cached view is still expired when red is modified by any
        # other code.
        red = Color.objects.get(slug="red")

        return context

Template:

{# variable "color" is the object "blue" #}
{% load ultracache_tags %}
{% ultracache 300 "color-info" color.pk %}
    {# expensive properties follow #}
    {{ color.compute_valid_hex_codes }}
    {{ color.name_in_all_languages }}
{% endultracache %}

Arbitrary Python:

from ultracache.utils import Ultracache

...

color_slug = request.GET["color_slug"]
uc = Ultracache(300, "another-identifier", color_slug)
if uc:
    codes = uc.cached
else:
    color = Color.objects.get(slug=color_slug)
    codes = color.compute_valid_hex_codes()
    uc.cache(codes)
print(codes)

View with more specific rules:

from ultracache.decorators import ultracache

# Serve different cached versions for anonymous and authenticated users. "request"
# is always in scope in the decorator.
@ultracache(300, "request.user.is_authenticated")
class MyView(TemplateView):
    template_name = "my_view.html"

    def get_context_data(self, **kwargs):
        context = super().get_context_data(**kwargs)
        context["show_private_messages"] = self.request.user.is_authenticated
        return context

Installation

1. Install or add django-ultracache to your Python path.

2. Add ultracache to your INSTALLED_APPS setting.

3. Add ultracache.middleware.UltraCacheMiddleware to your MIDDLEWARE setting. It is recommended to add it as one of the first entries.

4. Ensure django.template.context_processors.request is in the context processors setting.

Features

1. Caches template fragments, views, Django Rest Framework viewsets.

2. It takes the sites framework into consideration, allowing different caching per site.

3. Crucially, it is aware of model objects that are subjected to its caching. When an object is modified all affected cache key are automatically expired. This allows the user to set longer expiry times without having to worry about stale content.

4. The cache invalidation can be extended to issue purge commands to Varnish, Nginx or other reverse caching proxies.

Usage

from ultracache.decorators import cached_get, ultracache


class CachedView(TemplateView):
    template_name = "cached_view.html"

    @cached_get(300, "request.is_secure()", 456)
    def get(self, *args, **kwargs):
        return super(CachedView, self).get(*args, **kwargs)

@ultracache(300, "request.is_secure()", 456)
class AnotherCachedView(TemplateView):
    template_name = "cached_view.html"

from ultracache.decorators import cached_get

url(
    r"^cached-view/$",
    cached_get(3600)(TemplateView.as_view(
        template_name="myproduct/template.html"
    )),
    name="cached-view"
)

{% load ultracache_tags %}
{% ultracache 3600 "my_identifier" object 123 undefined "string" %}
    {{ object.title }}
{% endultracache %}

{% load ultracache_tags %}
{% ultracache 1200 "outer" %}
    {% ultracache 1200 "inner_one" %}
        title = {{ one.title }}
    {% endultracache %}
    {% ultracache 1200 "inner_two" %}
        title = {{ two.title }}
    {% endultracache %}
{% endultracache %}

# Cache all viewsets
ULTRACACHE = {
    "drf": {"viewsets": {"*": {}}}

}

# Cache a specific viewset by name
ULTRACACHE = {
    "drf": {"viewsets": {"my.app.MyViewset": {}}}

}

# Cache a specific viewset by class
ULTRACACHE = {
    "drf": {"viewsets": {MyViewset: {}}}

}

# Timeouts default to 300 seconds
ULTRACACHE = {
    "drf": {"viewsets": {"*": {"timeout": 1200}}}

}

# Evaluate code to append to the cache key. This example caches differently
# depending on whether the user is logged in or not.
ULTRACACHE = {
    "drf": {"viewsets": {"*": {"evaluate": "request.user.is_anonymous"}}}

}

# Evaluate code to append to the cache key via a callable.
def mycallable(viewset, request):
    if viewset.__class__.__name__ == "foo":
        return request.user.id

ULTRACACHE = {
    "drf": {"viewsets": {"*": {"evaluate": mycallable}}}

}

ULTRACACHE = {
    "purge": {"method": "myproduct.purgers.squid"}
}

virtualenv ve
./ve/bin/pip install -e .
./ve/bin/python bin/cache-purge-consumer.py -c config.yaml

ULTRACACHE = {
    "invalidate": False
}

ULTRACACHE = {
    "max-registry-value-size": 10000
}

ULTRACACHE = {
    "consider-headers": ["cookie"]
}

ULTRACACHE = {
    "consider-cookies": ["sessionid", "some-other-cookie"]
}

How does it work?

django-ultracache monkey patches django.template.base.Variable._resolve_lookup and django.db.models.Model.__getattribute__ to make a record of model objects as they are resolved. The ultracache template tag, ultracache decorator and ultracache context manager inspect the list of objects contained within them and keep a registry in Django’s caching backend. A post_save signal handler monitors objects for changes and expires the appropriate cache keys.

Tips

1. If you are running a cluster of Django nodes then ensure that they use a shared caching backend.

2.2

1. Django 4.0 compatibility.

2.1.1

1. Ensure cache coherency should a purger fail.

2.1.0

1. Django 3 compatibility.

2. Fix potential thread local residual data issue.

2.0.0

1. Remove dependency on the sites framework everywhere. The sites framework is still automatically considered if an installed app.

2. Do not store metadata in the request anymore but in a list on thread locals.

3. Introduce class utils.Ultracache to subject arbitrary pieces of Python code to caching.

4. Drop Django 1 support.

1.11.12

1. Simpler class based decorator.

2. Add Django 2.1 and Python 3.6 tests.

1.11.11

1. Add a test for tasks.

1.11.10

1. Ensure a working error message if pika is not found.

2. cached_get now considers any object accessed in get_context_data and not just objects accessed in the view template.

3. The original request headers are now sent to the purgers along with the path. This enables fine-grained proxy invalidation.

4. Django 2.0 and Python 3 compatibility. Django 1.9 support has been dropped.

1.11.9

1. Simplify the DRF caching implementation. It also now considers objects touched by sub-serializers.

1.11.8

1. The DRF settings now accept dotted names.

2. The DRF setting now accepts a callable whose result forms part of the cache key.

1.11.7

1. Use pickle to cache DRF data because DRF uses a Decimal type that isn’t recognized by Python’s json library.

1.11.6

1. Adjust the DRF decorator so it can be used in more places.

1.11.5

1. Django Rest Framework caching does not cache the entire response anymore, only the data and headers.

1.11.4

1. Move the twisted work to django-ultracache-twisted.

2. Clearly raise exception if libraries are not found.

1.11.3

1. Move the twisted directory one lower.

1.11.2

1. Package the product properly so all directories are included.

1.11.1

1. More defensive code to ensure we don’t interfere during migrations in a test run.

1.11.0

1. Introduce rabbitmq-url setting for use by broadcast_purge task.

2. Django 1.11 support.

3. Deprecate Django 1.6 support.

1.10.2

1. Remove logic that depends on SITE_ID so site can also be inferred from the request.

1.10.1

1. Add caching for Django Rest Framework viewsets.

2. Django 1.10 compatibility.

1.9.1

1. Add missing import only surfacing in certain code paths.

2. Invalidate setting was not being loaded properly. Fixed.

3. Handle content types RuntimeError when content types have not been migrated yet.

1.9.0

1. Move to tox for tests.

2. Django 1.9 compatibility.

0.3.8

1. Honor the raw parameter send along by loaddata. It prevents redundant post_save handling.

0.3.7

1. Revert the adding of the template name. It introduces a performance penalty in a WSGI environment.

2. Further reduce the number of writes to the cache.

0.3.6

1. Add template name (if possible) to the caching key.

2. Reduce number of calls to set_many.

0.3.5

1. Keep the metadata cache size in check to prevent possibly infinite growth.

0.3.4

1. Prevent redundant sets.

2. Work around an apparent Python bug related to di[k].append(v) vs di[k] = di[k] + [v]. The latter is safe.

0.3.3

1. Handle case where one cached view renders another cached view inside it, thus potentially sharing the same cache key.

0.3.2

1. The ultracache template tag now only caches HEAD and GET requests.

0.3.1

1. Trivial release to work around Pypi errors of the day.

0.3

1. Replace cache.get in for loop with cache.get_many.

0.2

1. Do not automatically add request.get_full_path() if any of request.get_full_path(), request.path or request.path_info is an argument for cached_get.

0.1.6

1. Also cache response headers.

0.1.5

1. Explicitly check for GET and HEAD request method and cache only those requests.

0.1.4

1. Rewrite decorator to be function based instead of class based so it is easier to use in urls.py.

0.1.3

1. cached_get decorator now does not cache if request contains messages.

0.1.2

1. Fix HTTPResponse caching bug.

0.1.1

1. Handle case where a view returns an HTTPResponse object.

0.1

1. Initial release.