Skip to main content

Improve performance and maintainability with a prefetching layer in your Django / Django REST Framework project

Project description

Django Virtual Models Icon

Django Virtual Models

Improve performance and maintainability with a prefetching layer in your Django / Django REST Framework project

Test   Coverage Status   Package version   Supported Python versions


Documentation: https://vintasoftware.github.io/django-virtual-models/

Example project: https://github.com/vintasoftware/django-virtual-models/tree/main/example

Source Code: https://github.com/vintasoftware/django-virtual-models


Django Virtual Models introduces a new "prefetching layer" to Django codebases that assists developers to express complex read logic without sacrificing maintainability, composability and performance. A Virtual Model allows developers to declare all nesting they need along with all annotations, prefetches, and joins in a single declarative class.

When implementing Django REST Framework serializers, developers need to be careful to avoid causing the N+1 selects problem due to missing prefetch_related or select_related calls on the associated queryset. Additionaly, developers must not miss annotate calls for fields that are computed at queryset-level.

With Virtual Models integration with DRF, if you change a DRF Serializer, you won't forget to modify the associated queryset with additional annotations, prefetches, and joins. If you do forget to update the queryset, Django Virtual Models will guide you by raising friendly exceptions to assist you to write the correct Virtual Model for the serializer you're changing. This guidance will prevent N+1s and missing annotations in all serializers that use Virtual Models.

For example, imagine if you have following nested serializers starting from MovieSerializer:

from movies.models import Nomination, Person, Movie

class AwardSerializer(serializers.ModelSerializer):
    class Meta:
        model = Nomination
        fields = ["award", "category", "year", "is_winner"]

class PersonSerializer(serializers.ModelSerializer):
    awards = AwardSerializer(many=True)
    nomination_count = serializers.IntegerField(read_only=True)

    class Meta:
        model = Person
        fields = ["name", "awards", "nomination_count"]

class MovieSerializer(serializers.ModelSerializer):
    directors = PersonSerializer(many=True)

    class Meta:
        model = Movie
        fields = ["name", "directors"]

For good performance and correct functionality, all nested serializers must have a corresponding prefetch_related on the queryset used by MovieSerializer. Also, the nomination_count field should be annotated on it. Therefore, you'll need to write this complex chain of nested prefetches:

from django.db.models import Prefetch

awards_qs = Nomination.objects.filter(is_winner=True)

directors_qs = Person.objects.prefetch_related(
    Prefetch(
        "nominations",
        queryset=awards_qs,
        to_attr="awards"
    )
).annotate(
    nomination_count=Count("nominations")
).distinct()

qs = Movie.objects.prefetch_related(
    Prefetch(
        "directors",
        queryset=directors_qs
    )
)

Conversely, you can declare Virtual Models for this read logic to easily reuse and customize those classes in multiple places of the codebase:

import django_virtual_models as v

class VirtualAward(v.VirtualModel):
    class Meta:
        model = Nomination

    def get_prefetch_queryset(self, **kwargs):
        return Nomination.objects.filter(is_winner=True)


class VirtualPerson(v.VirtualModel):
    awards = VirtualAward(lookup="nominations")
    nomination_count = v.Annotation(
        lambda qs, **kwargs: qs.annotate(
            nomination_count=Count("nominations")
        ).distinct()
    )

    class Meta:
        model = Person


class VirtualMovie(v.VirtualModel):
    directors = VirtualPerson()

    class Meta:
        model = Movie

To configure your DRF view and serializer to use Virtual Models, inherit from the proper classes:

import django_virtual_models as v

class MovieSerializer(v.VirtualModelSerializer):
    ...

    class Meta:
        ...
        virtual_model = VirtualMovie

class MovieList(v.VirtualModelListAPIView):
    queryset = Movie.objects.all()
    serializer_class = MovieSerializer
    ...

Then the library will automatically do the right prefetches and annotations for you!

If, for example, you forget to add the nomination_count field on VirtualPerson, the following exception will appear when using MovieSerializer:

MissingVirtualModelFieldException exception

If you aren't using DRF serializers, you hydrate your queryset with virtual fields manually:

qs = VirtualMovie().get_optimized_queryset(
    Movie.objects.all(),
    lookup_list=[
        "directors__awards",
        "directors__nomination_count",
    ]
)

To learn more, check the Installation and the Tutorial. Or the example project.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

django-virtual-models-0.2.0.tar.gz (53.3 kB view details)

Uploaded Source

Built Distribution

django_virtual_models-0.2.0-py3-none-any.whl (21.4 kB view details)

Uploaded Python 3

File details

Details for the file django-virtual-models-0.2.0.tar.gz.

File metadata

File hashes

Hashes for django-virtual-models-0.2.0.tar.gz
Algorithm Hash digest
SHA256 507429a8c48baf15f90c16d4be86fcecaaea1aef6cab5292582329c751a9b5de
MD5 68118ae78c897a23b84ef633f5608566
BLAKE2b-256 75652a1b5de610fe999eb992d7e3de22319dec637326bd0ea476de3c2b6871ce

See more details on using hashes here.

File details

Details for the file django_virtual_models-0.2.0-py3-none-any.whl.

File metadata

File hashes

Hashes for django_virtual_models-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 e1083abe14c383d7079fc104b0db23ecc306100902da6452b8a2c3f2861b7597
MD5 54df32ac92068e26cc012b1dade47219
BLAKE2b-256 92672dfefb810f483417f8d89bc8d3df4b859a6bb9450dc6be137f4a37a8f035

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page