Improve performance and maintainability with a prefetching layer in your Django / Django REST Framework project
Project description
Django Virtual Models
Improve performance and maintainability with a prefetching layer in your Django / Django REST Framework project
Documentation: https://vintasoftware.github.io/django-virtual-models/
Example project: https://github.com/vintasoftware/django-virtual-models/tree/main/example
Source Code: https://github.com/vintasoftware/django-virtual-models
Django Virtual Models introduces a new "prefetching layer" to Django codebases that assists developers to express complex read logic without sacrificing maintainability, composability and performance. A Virtual Model allows developers to declare all nesting they need along with all annotations, prefetches, and joins in a single declarative class.
When implementing Django REST Framework serializers, developers need to be careful to avoid causing the N+1 selects problem due to missing prefetch_related
or select_related
calls on the associated queryset. Additionaly, developers must not miss annotate
calls for fields that are computed at queryset-level.
With Virtual Models integration with DRF, if you change a DRF Serializer, you won't forget to modify the associated queryset with additional annotations, prefetches, and joins. If you do forget to update the queryset, Django Virtual Models will guide you by raising friendly exceptions to assist you to write the correct Virtual Model for the serializer you're changing. This guidance will prevent N+1s and missing annotations in all serializers that use Virtual Models.
For example, imagine if you have following nested serializers starting from MovieSerializer
:
from movies.models import Nomination, Person, Movie
class AwardSerializer(serializers.ModelSerializer):
class Meta:
model = Nomination
fields = ["award", "category", "year", "is_winner"]
class PersonSerializer(serializers.ModelSerializer):
awards = AwardSerializer(many=True)
nomination_count = serializers.IntegerField(read_only=True)
class Meta:
model = Person
fields = ["name", "awards", "nomination_count"]
class MovieSerializer(serializers.ModelSerializer):
directors = PersonSerializer(many=True)
class Meta:
model = Movie
fields = ["name", "directors"]
For good performance and correct functionality, all nested serializers must have a corresponding prefetch_related
on the queryset used by MovieSerializer
. Also, the nomination_count
field should be annotate
d on it. Therefore, you'll need to write this complex chain of nested prefetches:
from django.db.models import Prefetch
awards_qs = Nomination.objects.filter(is_winner=True)
directors_qs = Person.objects.prefetch_related(
Prefetch(
"nominations",
queryset=awards_qs,
to_attr="awards"
)
).annotate(
nomination_count=Count("nominations")
).distinct()
qs = Movie.objects.prefetch_related(
Prefetch(
"directors",
queryset=directors_qs
)
)
Conversely, you can declare Virtual Models for this read logic to easily reuse and customize those classes in multiple places of the codebase:
import django_virtual_models as v
class VirtualAward(v.VirtualModel):
class Meta:
model = Nomination
def get_prefetch_queryset(self, **kwargs):
return Nomination.objects.filter(is_winner=True)
class VirtualPerson(v.VirtualModel):
awards = VirtualAward(lookup="nominations")
nomination_count = v.Annotation(
lambda qs, **kwargs: qs.annotate(
nomination_count=Count("nominations")
).distinct()
)
class Meta:
model = Person
class VirtualMovie(v.VirtualModel):
directors = VirtualPerson()
class Meta:
model = Movie
To configure your DRF view and serializer to use Virtual Models, inherit from the proper classes:
import django_virtual_models as v
class MovieSerializer(v.VirtualModelSerializer):
...
class Meta:
...
virtual_model = VirtualMovie
class MovieList(v.VirtualModelListAPIView):
queryset = Movie.objects.all()
serializer_class = MovieSerializer
...
Then the library will automatically do the right prefetches and annotations for you!
If, for example, you forget to add the nomination_count
field on VirtualPerson
, the following exception will appear when using MovieSerializer
:
If you aren't using DRF serializers, you hydrate your queryset with virtual fields manually:
qs = VirtualMovie().get_optimized_queryset(
Movie.objects.all(),
lookup_list=[
"directors__awards",
"directors__nomination_count",
]
)
To learn more, check the Installation and the Tutorial. Or the example project.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file django-virtual-models-0.2.0.tar.gz
.
File metadata
- Download URL: django-virtual-models-0.2.0.tar.gz
- Upload date:
- Size: 53.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: python-requests/2.28.1
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 507429a8c48baf15f90c16d4be86fcecaaea1aef6cab5292582329c751a9b5de |
|
MD5 | 68118ae78c897a23b84ef633f5608566 |
|
BLAKE2b-256 | 75652a1b5de610fe999eb992d7e3de22319dec637326bd0ea476de3c2b6871ce |
File details
Details for the file django_virtual_models-0.2.0-py3-none-any.whl
.
File metadata
- Download URL: django_virtual_models-0.2.0-py3-none-any.whl
- Upload date:
- Size: 21.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: python-requests/2.28.1
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e1083abe14c383d7079fc104b0db23ecc306100902da6452b8a2c3f2861b7597 |
|
MD5 | 54df32ac92068e26cc012b1dade47219 |
|
BLAKE2b-256 | 92672dfefb810f483417f8d89bc8d3df4b859a6bb9450dc6be137f4a37a8f035 |