django-elasticsearch-dsl-drf

Integrate Elasticsearch DSL with Django REST framework.

These details have not been verified by PyPI

Project links

Homepage

Project description

Integrate django-elasticsearch-dsl with Django REST framework in the shortest way possible, with least efforts possible.

Package provides views, filter backends and other handy tools.

You are expected to use django-elasticsearch-dsl for defining your document models.

Prerequisites

Django 1.8, 1.9, 1.10 and 1.11.
Python 2.7, 3.4, 3.5, 3.6
Elasticsearch 2.x, 5.x

Dependencies

django-elasticsearch-dsl
djangorestframework

Installation

Install latest stable version from PyPI:

pip install django-elasticsearch-dsl-drf

Or latest stable version from GitHub:

pip install https://github.com/barseghyanartur/django-elasticsearch-dsl-drf/archive/stable.tar.gz

Add rest_framework and django_elasticsearch_dsl to INSTALLED_APPS:

INSTALLED_APPS = (
    # ...
    'rest_framework',  # REST framework
    'django_elasticsearch_dsl',  # ElasticSearch integration
    # ...
)

Usage examples

See the example project for sample models/views/serializers.

Basic Django REST framework integration example

Sample basic example models

books/models.py:

class Publisher(models.Model):
    """Publisher."""

    name = models.CharField(max_length=30)
    address = models.CharField(max_length=50)
    city = models.CharField(max_length=60)
    state_province = models.CharField(max_length=30)
    country = models.CharField(max_length=50)
    website = models.URLField()

    class Meta(object):
        """Meta options."""

        ordering = ["id"]

    def __str__(self):
        return self.name

Sample basic example document

search_indexes/documents/publisher.py:

from django_elasticsearch_dsl import DocType, Index, fields
from elasticsearch_dsl import analyzer

from books.models import Publisher

# Name of the ElasticSearch index
PUBLISHER_INDEX = Index('publisher')
# See ElasticSearch Indices API reference for available settings
PUBLISHER_INDEX.settings(
    number_of_shards=1,
    number_of_replicas=1
)


@PUBLISHER_INDEX.doc_type
class PublisherDocument(DocType):
    """Publisher ElasticSearch document."""

    id = fields.IntegerField(attr='id')

    name = fields.StringField(
        fields={
            'raw': fields.StringField(
                analyzer='keyword'
            )
        }
    )
    address = fields.StringField(
        fields={
            'raw': fields.StringField(
                analyzer='keyword'
            )
        }
    )
    city = fields.StringField(
        fields={
            'raw': fields.StringField(
                analyzer='keyword'
            )
        }
    )
    state_province = fields.StringField(
        fields={
            'raw': fields.StringField(
                analyzer='keyword'
            )
        }
    )
    country = fields.StringField(
        fields={
            'raw': fields.StringField(
                analyzer='keyword'
            )
        }
    )
    website = fields.StringField(
        fields={
            'raw': fields.StringField(
                analyzer='keyword'
            )
        }
    )

    class Meta(object):
        """Meta options."""

        model = Publisher  # The model associate with this DocType

Sample basic example serializer

search_indexes/serializers.py:

import json

from rest_framework import serializers

class PublisherDocumentSerializer(serializers.Serializer):
    """Serializer for Publisher document."""

    id = serializers.IntegerField(read_only=True)
    name = serializers.CharField(read_only=True)
    address = serializers.CharField(read_only=True)
    city = serializers.CharField(read_only=True)
    state_province = serializers.CharField(read_only=True)
    country = serializers.CharField(read_only=True)
    website = serializers.CharField(read_only=True)

    class Meta(object):
        """Meta options."""

        fields = read_only_fields = (
            'id',
            'name',
            'address',
            'city',
            'state_province',
            'country',
            'website',
        )
        read_only_fields = fields

Sample basic example view

search_indexes/views.py:

from django_elasticsearch_dsl_drf.constants import (
    LOOKUP_FILTER_TERMS,
    LOOKUP_FILTER_RANGE,
    LOOKUP_FILTER_PREFIX,
    LOOKUP_FILTER_WILDCARD,
    LOOKUP_QUERY_IN,
    LOOKUP_QUERY_EXCLUDE,
)
from django_elasticsearch_dsl_drf.filter_backends import (
    FilteringFilterBackend,
    OrderingFilterBackend,
    SearchFilterBackend,
)
from django_elasticsearch_dsl_drf.views import BaseDocumentViewSet

# Example app models
from search_indexes.documents.publisher import PublisherDocument
from search_indxes.serializers import PublisherDocumentSerializer

class PublisherDocumentView(BaseDocumentViewSet):
    """The PublisherDocument view."""

    document = PublisherDocument
    serializer_class = PublisherDocumentSerializer
    lookup_field = 'id'
    filter_backends = [
        FilteringFilterBackend,
        OrderingFilterBackend,
        SearchFilterBackend,
    ]
    # Define search fields
    search_fields = (
        'name',
        'address',
        'city',
        'state_province',
        'country',
    )
    # Define filtering fields
    filter_fields = {
        'id': None,
        'name': 'name.raw',
        'city': 'city.raw',
        'state_province': 'state_province.raw',
        'country': 'country.raw',
    }
    # Define ordering fields
    ordering_fields = {
        'id': None,
        'name': None,
        'city': None,
        'country': None,
    }
    # Specify default ordering
    ordering = ('id', 'name',)

Advanced Django REST framework integration example

Sample advanced example models

books/models.py:

import json

from django.conf import settings
from django.db import models
from django.utils.translation import ugettext, ugettext_lazy as _

from six import python_2_unicode_compatible

BOOK_PUBLISHING_STATUS_PUBLISHED = 'published'
BOOK_PUBLISHING_STATUS_NOT_PUBLISHED = 'not_published'
BOOK_PUBLISHING_STATUS_IN_PROGRESS = 'in_progress'
BOOK_PUBLISHING_STATUS_CANCELLED = 'cancelled'
BOOK_PUBLISHING_STATUS_REJECTED = 'rejected'
BOOK_PUBLISHING_STATUS_CHOICES = (
    (BOOK_PUBLISHING_STATUS_PUBLISHED, "Published"),
    (BOOK_PUBLISHING_STATUS_NOT_PUBLISHED, "Not published"),
    (BOOK_PUBLISHING_STATUS_IN_PROGRESS, "In progress"),
    (BOOK_PUBLISHING_STATUS_CANCELLED, "Cancelled"),
    (BOOK_PUBLISHING_STATUS_REJECTED, "Rejected"),
)
BOOK_PUBLISHING_STATUS_DEFAULT = BOOK_PUBLISHING_STATUS_PUBLISHED


@python_2_unicode_compatible
class Publisher(models.Model):
    """Publisher."""

    name = models.CharField(max_length=30)
    address = models.CharField(max_length=50)
    city = models.CharField(max_length=60)
    state_province = models.CharField(max_length=30)
    country = models.CharField(max_length=50)
    website = models.URLField()

    class Meta(object):
        """Meta options."""

        ordering = ["id"]

    def __str__(self):
        return self.name


@python_2_unicode_compatible
class Author(models.Model):
    """Author."""

    salutation = models.CharField(max_length=10)
    name = models.CharField(max_length=200)
    email = models.EmailField()
    headshot = models.ImageField(upload_to='authors', null=True, blank=True)

    class Meta(object):
        """Meta options."""

        ordering = ["id"]

    def __str__(self):
        return self.name


class Tag(models.Model):
    """Simple tag model."""

    title = models.CharField(max_length=255, unique=True)

    class Meta(object):
        """Meta options."""

        verbose_name = _("Tag")
        verbose_name_plural = _("Tags")

    def __str__(self):
        return self.title


@python_2_unicode_compatible
class Book(models.Model):
    """Book."""

    title = models.CharField(max_length=100)
    description = models.TextField(null=True, blank=True)
    summary = models.TextField(null=True, blank=True)
    authors = models.ManyToManyField('books.Author', related_name='books')
    publisher = models.ForeignKey(Publisher, related_name='books')
    publication_date = models.DateField()
    state = models.CharField(max_length=100,
                             choices=BOOK_PUBLISHING_STATUS_CHOICES,
                             default=BOOK_PUBLISHING_STATUS_DEFAULT)
    isbn = models.CharField(max_length=100, unique=True)
    price = models.DecimalField(max_digits=10, decimal_places=2)
    pages = models.PositiveIntegerField(default=200)
    stock_count = models.PositiveIntegerField(default=30)
    tags = models.ManyToManyField('books.Tag',
                                  related_name='books',
                                  blank=True)

    class Meta(object):
        """Meta options."""

        ordering = ["isbn"]

    def __str__(self):
        return self.title

    @property
    def publisher_indexing(self):
        """Publisher for indexing.

        Used in ElasticSearch indexing.
        """
        if self.publisher is not None:
            return self.publisher.name

    @property
    def tags_indexing(self):
        """Tags for indexing.

        Used in ElasticSearch indexing.
        """
        return json.dumps([tag.title for tag in self.tags.all()])

Sample advanced example document

search_indexes/documents/book.py:

from django_elasticsearch_dsl import DocType, Index, fields
from elasticsearch_dsl import analyzer

from books.models import Book

# Name of the ElasticSearch index
BOOK_INDEX = Index('book')
# See ElasticSearch Indices API reference for available settings
BOOK_INDEX.settings(
    number_of_shards=1,
    number_of_replicas=1
)


html_strip = analyzer(
    'html_strip',
    tokenizer="standard",
    filter=["standard", "lowercase", "stop", "snowball"],
    char_filter=["html_strip"]
)


@BOOK_INDEX.doc_type
class BookDocument(DocType):
    """Book ElasticSearch document."""

    id = fields.IntegerField(attr='id')

    title = fields.StringField(
        analyzer=html_strip,
        fields={
            'raw': fields.StringField(
                analyzer='keyword'
            )
        }
    )

    description = fields.StringField(
        analyzer=html_strip,
        fields={
            'raw': fields.StringField(
                analyzer='keyword'
            )
        }
    )

    summary = fields.StringField(
        analyzer=html_strip,
        fields={
            'raw': fields.StringField(
                analyzer='keyword'
            )
        }
    )

    publisher = fields.StringField(
        attr='publisher_indexing',
        analyzer=html_strip,
        fields={
            'raw': fields.StringField(
                analyzer='keyword'
            )
        }
    )

    publication_date = fields.DateField()

    state = fields.StringField(
        analyzer=html_strip,
        fields={
            'raw': fields.StringField(
                analyzer='keyword'
            )
        }
    )

    isbn = fields.StringField(
        analyzer=html_strip,
        fields={
            'raw': fields.StringField(
                analyzer='keyword'
            )
        }
    )

    price = fields.FloatField()

    pages = fields.IntegerField()

    stock_count = fields.IntegerField()

    tags = fields.StringField(
        attr='tags_indexing',
        analyzer=html_strip,
        fields={
            'raw': fields.StringField(
                analyzer='keyword',
                multi=True
            )
        },
        multi=True
    )

    class Meta(object):
        """Meta options."""

        model = Book  # The model associate with this DocType

Sample advanced example serializer

search_indexes/serializers.py:

import json

from rest_framework import serializers

class TagSerializer(serializers.Serializer):
    """Helper serializer for the Tag field of the Book document."""

    title = serializers.CharField()

    class Meta(object):
        """Meta options."""

        fields = ('title',)
        read_only_fields = ('title',)


class BookDocumentSerializer(serializers.Serializer):
    """Serializer for the Book document."""

    id = serializers.SerializerMethodField()

    title = serializers.CharField(read_only=True)
    description = serializers.CharField(read_only=True)
    summary = serializers.CharField(read_only=True)

    publisher = serializers.CharField(read_only=True)
    publication_date = serializers.DateField(read_only=True)
    state = serializers.CharField(read_only=True)
    isbn = serializers.CharField(read_only=True)
    price = serializers.FloatField(read_only=True)
    pages = serializers.IntegerField(read_only=True)
    stock_count = serializers.IntegerField(read_only=True)
    tags = serializers.SerializerMethodField()

    class Meta(object):
        """Meta options."""

        fields = (
            'id',
            'title',
            'description',
            'summary',
            'publisher',
            'publication_date',
            'state',
            'isbn',
            'price',
            'pages',
            'stock_count',
            'tags',
        )
        read_only_fields = (
            'id',
            'title',
            'description',
            'summary',
            'publisher',
            'publication_date',
            'state',
            'isbn',
            'price',
            'pages',
            'stock_count',
            'tags',
        )

    def get_tags(self, obj):
        """Get tags."""
        return json.loads(obj.tags)

Sample advanced example view

search_indexes/views.py:

from django_elasticsearch_dsl_drf.constants import (
    LOOKUP_FILTER_TERMS,
    LOOKUP_FILTER_RANGE,
    LOOKUP_FILTER_PREFIX,
    LOOKUP_FILTER_WILDCARD,
    LOOKUP_QUERY_IN,
    LOOKUP_QUERY_EXCLUDE,
)
from django_elasticsearch_dsl_drf.filter_backends import (
    FilteringFilterBackend,
    OrderingFilterBackend,
    SearchFilterBackend,
)
from django_elasticsearch_dsl_drf.views import BaseDocumentViewSet

# Example app models
from search_indexes.documents.book import BookDocument
from search_indxes.serializers import BookDocumentSerializer


class BookDocumentView(BaseDocumentViewSet):
    """The BookDocument view."""

    document = BookDocument
    serializer_class = BookDocumentSerializer
    lookup_field = 'id'
    filter_backends = [
        FilteringFilterBackend,
        OrderingFilterBackend,
        SearchFilterBackend,
    ]
    # Define search fields
    search_fields = (
        'title',
        'description',
        'summary',
    )
    # Define filtering fields
    filter_fields = {
        'id': {
            'field': '_id',
            'lookups': [
                LOOKUP_FILTER_RANGE,
                LOOKUP_QUERY_IN,
            ],
        },
        'publisher': {
            'field': 'publisher.raw',
        },
        'publication_date': 'publication_date',
        'isbn': {
            'field': 'isbn.raw',
        },
        'tags': {
            'field': 'tags',
            'lookups': [
                LOOKUP_FILTER_TERMS,
                LOOKUP_FILTER_PREFIX,
                LOOKUP_FILTER_WILDCARD,
                LOOKUP_QUERY_IN,
                LOOKUP_QUERY_EXCLUDE,
            ],
        },
        'tags.raw': {
            'field': 'tags.raw',
            'lookups': [
                LOOKUP_FILTER_TERMS,
                LOOKUP_FILTER_PREFIX,
                LOOKUP_FILTER_WILDCARD,
                LOOKUP_QUERY_IN,
                LOOKUP_QUERY_EXCLUDE,
            ],
        },
    }
    # Define ordering fields
    ordering_fields = {
        'id': 'id',
        'title': 'title.raw',
        'price': 'price.raw',
        'state': 'state.raw',
        'publication_date': 'publication_date',
    }
    # Specify default ordering
    ordering = ('id', 'title',)

Usage

Considering samples above, you should be able to perform the search, sorting and filtering actions described below.

Sample queries

Search

Query param name reserved for search is search. Make sure your models and documents do not have it as a field or attribute.

Multiple search terms are joined with OR.

Let’s assume we have a number of Book items with fields title`, ``description and summary.

Search in all fields

Search in all fields (title, summary and content) for word “education”

http://127.0.0.1:8080/search/books/?search=education

Search in specific field

In order to search in specific field (title) for term “education”, add the field name separated with | to the search term.

http://127.0.0.1:8080/search/books/?search=title|education

Search for multiple terms

In order to search for multiple terms “education”, “technology” add multiple search query params.

http://127.0.0.1:8080/search/books/?search=education&search=technology

Search for multiple terms in specific fields

In order to search for multiple terms “education”, “technology” in specific fields add multiple search query params and field names separated with | to each of the search terms.

http://127.0.0.1:8080/search/books/?search=title|education&search=summary|technology

Filtering

Let’s assume we have a number of Book documents with the tags (education, politics, economy, biology, climate, environment, internet, technology).

Multiple filter terms are joined with AND.

Filter documents by state

Filter documents by state “published”.

http://127.0.0.1:8080/search/books/?state=published

Filter documents by multiple states

Filter documents by states “published” and “in_progress”

http://127.0.0.1:8080/search/books/?state__in=published|in_progress

Filter document by a single field

Filter documents by (field tag) “education”.

http://127.0.0.1:8080/search/books/?tag=education

Filter documents by multiple fields

Filter documents by multiple fields (field tags) “education” and “economy” with use of functional in query filter.

http://127.0.0.1:8080/search/books/?tags__in=education|economy

You can achieve the same effect by specifying multiple tags (tags) “education” and “economy”. Note, that in this case multiple filter terms are joined with OR.

http://127.0.0.1:8080/search/books/?tags=education&tags=economy

If you want the same as above, but joined with AND, add __term to each lookup.

http://127.0.0.1:8080/search/books/?tags__term=education&tags__term=economy

Filter documents by a word part of a single field

Filter documents by a part word part in single field (tags). Word part should match both “technology” and “biology”.

http://127.0.0.1:8080/search/books/?tags__wildcard=*logy

Ordering

The - prefix means ordering should be descending.

Order documents by field (ascending)

Filter documents by field price (ascending).

http://127.0.0.1:8080/search/books/?search=title|lorem&ordering=price

Order documents by field (descending)

Filter documents by field price (descending).

http://127.0.0.1:8080/search/books/?search=title|lorem&ordering=-price

Order documents by multiple fields

If you want to order by multiple fields, use multiple ordering query params. In the example below, documents would be ordered first by field publication_date (descending), then by field price (ascending).

http://127.0.0.1:8080/search/books/?search=title|lorem&ordering=-publication_date,ordering=price

Various handy helpers

Testing

Project is covered with tests.

To test with all supported Python/Django versions type:

tox

To test against specific environment, type:

tox -e py36-django110

To test just your working environment type:

./runtests.py

To run a single test in your working environment type:

./runtests.py src/django_elasticsearch_dsl_drf/tests/test_ordering.py

Or:

./manage.py test django_elasticsearch_dsl_drf.tests.test_ordering

It’s assumed that you have all the requirements installed. If not, first install the test requirements:

pip install -r examples/requirements/test.txt

Writing documentation

Keep the following hierarchy.

=====
title
=====

header
======

sub-header
----------

sub-sub-header
~~~~~~~~~~~~~~

sub-sub-sub-header
^^^^^^^^^^^^^^^^^^

sub-sub-sub-sub-header
++++++++++++++++++++++

sub-sub-sub-sub-sub-header
**************************

License

GPL 2.0/LGPL 2.1

Support

For any issues contact me at the e-mail given in the Author section.

Author

Artur Barseghyan <artur.barseghyan@gmail.com>

Project details

These details have not been verified by PyPI

Project links

Homepage

Release history Release notifications | RSS feed

0.22.5

Jul 3, 2022

0.22.4

Jan 24, 2022

0.22.3

Jan 24, 2022

0.22.2

Aug 29, 2021

0.22.1

May 1, 2021

0.22

Mar 25, 2021

0.21

Feb 4, 2021

0.20.9

Oct 15, 2020

0.20.8

Apr 10, 2020

0.20.7

Apr 10, 2020

0.20.6

Apr 4, 2020

0.20.5

Dec 29, 2019

0.20.4

Dec 25, 2019

0.20.3

Sep 20, 2019

0.20.2

Aug 30, 2019

0.20.1

Aug 17, 2019

0.20

Aug 16, 2019

0.19

Aug 6, 2019

0.18

Jun 26, 2019

0.17.7

May 29, 2019

0.17.6

Apr 8, 2019

0.17.5

Apr 2, 2019

0.17.4

Mar 13, 2019

0.17.3

Feb 8, 2019

0.17.2

Jan 7, 2019

0.17.1

Dec 13, 2018

0.17

Dec 11, 2018

0.16.3

Oct 30, 2018

0.16.2

Sep 21, 2018

0.16.1

Sep 18, 2018

0.16

Sep 10, 2018

0.15.1

Aug 21, 2018

0.15

Aug 10, 2018

0.14

Aug 6, 2018

0.13.2

Aug 3, 2018

0.13.1

Jul 26, 2018

0.13

Jul 22, 2018

0.12

Jul 21, 2018

0.11

Jul 15, 2018

0.10

Jul 6, 2018

0.9

Jul 4, 2018

0.8.4

Jun 27, 2018

0.8.3

Jun 25, 2018

0.8.2

Jun 5, 2018

0.8.1

Jun 5, 2018

0.8

Jun 1, 2018

0.7.2

May 10, 2018

0.7.1

Apr 5, 2018

0.7

Mar 8, 2018

0.6.4

Mar 6, 2018

0.6.3

Jan 3, 2018

0.6.2

Dec 29, 2017

0.6.1

Nov 29, 2017

0.6

Nov 29, 2017

0.5.1

Oct 18, 2017

0.5

Oct 5, 2017

0.4.4

Oct 2, 2017

0.4.3

Sep 28, 2017

0.4.2

Sep 27, 2017

0.4.1

Sep 26, 2017

0.4

Sep 26, 2017

0.3.12

Sep 21, 2017

0.3.11

Sep 21, 2017

0.3.10

Sep 20, 2017

0.3.9

Sep 12, 2017

0.3.8

Sep 12, 2017

0.3.7

Sep 7, 2017

0.3.6

Sep 7, 2017

0.3.5

Aug 23, 2017

0.3.4

Aug 23, 2017

0.3.3

Jul 16, 2017

0.3.2

Jul 12, 2017

0.3.1

Jul 11, 2017

0.3

Jul 11, 2017

0.2.6

Jul 11, 2017

0.2.5

Jul 11, 2017

0.2.4

Jul 11, 2017

0.2.3

Jul 11, 2017

0.2.2

Jul 11, 2017

0.2.1

Jul 11, 2017

0.2

Jul 11, 2017

0.1.8

Jun 27, 2017

0.1.7

Jun 25, 2017

0.1.6

Jun 22, 2017

0.1.5

Jun 22, 2017

0.1.4

Jun 21, 2017

0.1.3

Jun 20, 2017

0.1.2

Jun 20, 2017

This version

0.1.1

Jun 20, 2017

0.1

Jun 20, 2017

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

django-elasticsearch-dsl-drf-0.1.1.tar.gz (45.8 kB view hashes)

Uploaded Jun 20, 2017 Source

Built Distribution

django_elasticsearch_dsl_drf-0.1.1-py2.py3-none-any.whl (39.5 kB view hashes)

Uploaded Jun 20, 2017 Python 2 Python 3

Hashes for django-elasticsearch-dsl-drf-0.1.1.tar.gz

Hashes for django-elasticsearch-dsl-drf-0.1.1.tar.gz
Algorithm	Hash digest
SHA256	`be321ac089544153773fc35a474417b959658cc012f4371d18a1425bac5d4f95`
MD5	`7f46b765aa14885d3018980fb690b3c4`
BLAKE2b-256	`2ef6538a22d285f613d33ff8ff059b8cb5e9b07944e23b52e69e153f18a5e394`

Hashes for django_elasticsearch_dsl_drf-0.1.1-py2.py3-none-any.whl

Hashes for django_elasticsearch_dsl_drf-0.1.1-py2.py3-none-any.whl
Algorithm	Hash digest
SHA256	`b9548f996c30c1174096e157c9870ef2a02cabfa5ba072a39e7c57c7517ba5d8`
MD5	`4b7666668c3c58c11b02587dd4bf8807`
BLAKE2b-256	`4a6db827232769007c28e57393999f53c0973a786502a8ed6b9fb5ae99de6549`

django-elasticsearch-dsl-drf 0.1.1

Navigation

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Project description

Prerequisites

Dependencies

Installation

Usage examples

Basic Django REST framework integration example

Sample basic example models

Sample basic example document

Sample basic example serializer

Sample basic example view

Advanced Django REST framework integration example

Sample advanced example models

Sample advanced example document

Sample advanced example serializer

Sample advanced example view

Usage

Sample queries

Search

Filtering

Ordering

Various handy helpers

Testing

Writing documentation

License

Support

Author

Project details

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Release history Release notifications | RSS feed

Download files

Source Distribution

Built Distribution