django-data-browser

Run pip install django-data-browser.
Add "data_browser" to installed_apps.
Add path("data-browser/", include("data_browser.urls")) to your urls.
Run python manage.py migrate.
If you have queryset annotations in your admin or are interested in exposing calculated values see the Calculated and Annotated fields section.

Settings

Name	Default	Docs Section	Function
DATA_BROWSER_ALLOW_PUBLIC	False	Security	Allow selected saved views to be accessed without admin login in limited circumstances.
DATA_BROWSER_AUTH_USER_COMPAT	True	Performance	When calling get_fieldsets on a UserAdmin always pass an instance of the associated model.
DATA_BROWSER_DEFAULT_ROW_LIMIT	1000		The default value for the row limit selector in the UI.
DATA_BROWSER_DEV	False	CONTRIBUTING.rst	Enable proxying frontend to JS dev server.
DATA_BROWSER_FE_DSN	None	Sentry	The DSN the frontend sentry should report to, disabled by default.

Most of the Django views in the Data Browser can only be accessed by Django “staff members”. These views support general querying of the database, checked against the admin permissions of the logged in user.

The only exception to this is “Public Saved Views” these are views which have been saved and marked as public. They can be accessed by anyone without needing a login but they can only be used to access a query that has been saved and made public and they have long random URL’s.

You can use the admin permission data_browser | view | Can make a saved view publically available to restrict who can make views public. To be public the view must be marked as public and owned by someone who has the permission.

Additionally the entire public views system is gated by the Django settings value DATA_BROWSER_ALLOW_PUBLIC.

Sentry

The frontend code has builtin Sentry support, it is disabled by default. To enable it set the Django settings value DATA_BROWSER_FE_DSN, for example to set it to the Data Browser project Sentry use:

DATA_BROWSER_FE_DSN = "https://af64f22b81994a0e93b82a32add8cb2b@o390136.ingest.sentry.io/5231151"

Linking to the Data Browser

The home page URL of the Data Browser is given by reverse("data_browser:home").

Additionally if you are using data_browser.helpers.AdminMixin then in Admin list views the URL of the Data Browser page for the same model is available as the template context variable ddb_url.

One convenient way of utilizing this is to create the file templates/admin/change_list_object_tools.html and populate it with:

{% extends "admin/change_list_object_tools.html" %}
{% block object-tools-items %}
    {{ block.super }}
    {% if ddb_url %}
        <li><a href="{{ ddb_url }}" class="viewlink">Data Browser</a></li>
    {% endif %}
{% endblock %}

This will place a “Data Browser” button on the list view of every admin that inherits from the mixin. Note: to do this at the top level the app you put the template in must be before contrib.admin in INSTALLED_APPS.

Specifying models and fields

By default the Data Browser has access to all models and fields that the current user can see anywhere in the Admin site. However if necessary this can be tweaked using the following class level properties and functions on ModelAdmins and Inlines.

Name	Format	Purpose
ddb_ignore get_ddb_ignore(request)	bool	Ignore this Admin / Inline entirely, will still show fields from other Inlines / Admins on the same model.
ddb_hide_fields get_ddb_hide_fields(request)	[field_name]	Explicitly hide the specified fields.
ddb_extra_fields get_ddb_extra_fields(request)	[field_name]	Add additional fields that are not mentioned in fields, fieldsets or list_display.
ddb_json_fields get_ddb_json_fields(request)	{field_name: {json_field_name: type}}	Expose fields within JSON data for access in the Data Browser. Type can be “string”, “number” or “boolean”.
ddb_default_filters get_ddb_default_filters()	[(path, lookup, value)]	Default filters to be added when opening this model. E.G. to add client__name__equals=Test use [("client__name", "equals", "Test")].

Additionally, per the below sections, calculated fields and actions can be hidden by setting the ddb_hide attribute and annotated fields are always visible unless explicitly hidden.

Calculated and Annotated fields

Calculated

Calculated fields are fields on the ModelAdmin whose value comes from a function on the ModelAdmin or a function or property on the Model itself, as described at the bottom of the Django admin list display docs.

Being arbitrary Python code calculated fields are opaque to the Data Browser. It can fetch their values but can’t sort or filter etc on them. For pivoting they are treated as equivalent to the pk on the same model.

Additionally calculated fields can be hidden from the Data Browser by setting the attribute ddb_hide to True. The data_browser.helpers.attributes decorator can make this a little tidier.

@attributes(ddb_hide=True)
def my_calculated_field(self, obj):
    return ...

Annotated

The Data Browser has additional support for annotated fields. Normally you would expose these as calculated fields. The module data_browser.helpers contains helpers which will make exposing annotated fields simpler, more performant and expose them to the Data Browser so it can do arbitrary manipulation with them.

Exposing an a annotated field in this way requires two changes.

Mix data_browser.helpers.AdminMixin into your ModelAdmin.
Add a function decorated with data_browser.helpers.annotation that takes and updates a queryset.

from data_browser.helpers import AdminMixin, annotation

@admin.register(MyModel)
class MyAdmin(AdminMixin, ModelAdmin):
    fields = ["my_field"]

    @annotation
    def my_field(self, request, qs):
        return qs.annotate(my_field=Cast(..., output_field=IntegerField()))

WARNING: annotated aggregations will produce misleading results when further aggregated in the Data Browser.

It is important that the decorated annotation function name and the annotated queryset field name match.

Sometimes it is necessary for the top level of the annotation to have output_field set so the Data Browser can tell what type of data it will produce. When this is necessary you will get an error to that effect.

The helpers will automatically deal with the admin_order_field and boolean properties and readonly_fields, reducing the boiler plate involved in using annotations in the admin.

Additionally the annotation will only be applied to the list view when it’s mentioned in list_display this allows you to use annotations extensively on your detail views without hurting the performance of your list views.

And finally even if not mentioned in fields, fieldsets or list_display, the annotation will still be visible in the Data Browser unless it is explicitly mentioned in ddb_hide_fields.

Performance

get_queryset

The Data Browser does it’s fetching in two stages.

First it does a single DB query to get the majority of the data. To construct the queryset for this it will call get_queryset on the ModelAdmin of the current Model. It uses .values() to fetch only the data it needs from the database and it will inline all referenced models to ensure it doesn’t do multiple queries.

At this stage annotated fields on related models are attached with subquery annotations, the data_browser will call get_queryset on the relevant ModelAdmins in order to generate these subquery annotations.

Secondly for any calculated fields it will then fetch the complete objects that are needed for those calculated fields. To construct the querysets for these it will call get_queryset on their associated ModelAdmins. These calls are aggregated so it will only make one per model.

As a simple example. If you did a query against the Book model for the fields:

book.name
book.author.name
book.author.age
book.author.number_of_books
book.publisher.name

Where the author.age is actually a property on the Author Model and author.number_of_books is an @annotation on the Author Admin then it would do something like the following two queries:

BookAdmin.get_queryset().annotate(
    author__number_of_books=Subquery(
        AuthorAdmin.get_queryset()
        .filter(pk=OuterRef("author__id"))
        .values("number_of_books")[:1]
    )
).values(
    "name",
    "author__name",
    "author__id",
    "author__number_of_books",
    "publisher__name",
)
AuthorAdmin.get_queryset().in_bulk(pks=...)

Where the pks passed to in_bulk in the second query came from author__id in the first.

When the Data Browser calls the admin get_queryset functions it will put some context in request.data_browser. This allows you to test to see if the Data Browser is making the call as follows:

if getattr(request, "data_browser"):
    # Data Browser specific customization

This is particularly useful if you want to route the Data Browser to a DB replica.

The context also includes a fields member that lists all the fields the Data Browser plans to access. You can use this to do conditional prefetching or annotating to support those fields like this:

if (
    not hasattr(request, "databrowser")
    or "my_field" in request.data_browser["fields"]
):
    # do prefetching and annotating associated with my_field

The AdminMixin described in the Calculated and Annotated fields section is doing this internally for @annotation fields.

get_fieldsets

The Data Browser also calls get_fieldsets to find out what fields the current user can access.

As with get_queryset the Data Browser will set request.data_browser when calling get_fieldsets and you can test this to detect it and make Data Browser specific customizations.

The Django User Admin has code to change the fieldsets when adding a new user. To compensate for this, when calling get_fieldsets on a subclass of django.contrib.auth.admin.UserAdmin the Data Browser will pass a newly constructed instance of the relevant model. This behavior can be disabled by setting settings.DATA_BROWSER_AUTH_USER_COMPAT to False.

Style customization

You can override the data_browser/index.html template per https://docs.djangoproject.com/en/3.1/howto/overriding-templates/#extending-an-overridden-template (make sure data_browser is after your app in INSTALLED_APPS) and replace the extrahead block.

This will let you inject custom CSS and stylesheets.

However note that because of how the normal CSS is built into the JS any custom CSS will be before the normal CSS so you will need to use more specific selectors or !important.

Version numbers

The Data Browser uses the standard Major.Minor.Patch version numbering scheme.

Patch versions may include bug fixes and minor features.

Minor versions are for significant new features.

Major versions are for major features, significant changes to existing functionality and breaking changes.

Patch and Minor versions should never contain breaking changes and should always be backward compatible. A breaking change is a change that makes backward incompatible changes to one or more of the following:

The query URL format.
The json, csv etc data formats, this does not include the Data Browsers internal API’s, only the data export formats.
The format of the request.data_browser passed to get_fieldsets and get_queryset.
Existing saved views.
The URL’s of public saved views.

For alpha and beta releases absolutely anything may change / break.

Release History

Version	Date	Summary
3.2.5	2020-02-07	Date filter values formated as 2020-1-2 are now considered ISO ordered and no longer ambiguous. Rework @annotation and AdminMixin so @annotation can be used on mixins.
3.2.4	2020-02-02	Fix equals and not equals not working for array fields. Improve date and datetime filter errors. Improve and contrast display of null and empty string. Various fixes for models where the primary key is not id. Empty but non null file fields render as empty string instead of null. Fix is null not working with the year function. The field list is now sorted by display name (except for the primary key and admin link). Fix not equals excluding nulls with functions and aggregates, e.g. year, min etc. Right click filter and drill down now correctly handle null values. Prevent exception when a saved views name gets too long.
3.2.3	2020-01-11	Fix issue when using a filter with a different type from the field, e.g. is null.
3.2.2	2020-12-30	Fix id field missing from some models. Per Django, Django 2.0 & 2.1 are not supported on Py3.8 and 3.9.
3.2.1	2020-12-30	Protect model admin class option values from accidental modification.
3.2.0	2020-12-30	Support for invoking admin actions by right clicking on id column headers. Fix various filter issues. Don’t show id on models that don’t have an id field. Show “less than”, “greater than” etc as “<”, “>”, etc. Mouse hover tooltip help for date and datetime filter values. Filters with bad fields and lookups are reported as errors rather than being ignored. Bad filters on public saved View’s now result in a 400 when loading the public URL. Fix issue filtering on aggregated annotations.
3.1.4	2020-12-19	Fix UUID’s not being filterable. Fix right click drill and filter trying to filter unfilterable fields. Fix spurious 0 appearing below numeric 0 filter values. Add an extrahead block to the template and documentation for overriding CSS.
3.1.3	2020-12-13	Relative time support in date and time filters. Show parsed dates and datetimes next to filters. Add view SQL link on front page.
3.1.2	2020-12-09	Remove length function from UUID’s. FK’s with no admin are exposed as just the FK field.
3.1.1	2020-12-01	Don’t run the 3.0.0 data migration when there are no saved views.
3.1.0	2020-11-29	Add right click menu with filter and drill down options.
3.0.4	2020-11-28	Ignore admins for things that are not Models.
3.0.3	2020-11-22	Fix exception when filtering to out of bounds year values.
3.0.2	2020-11-18	Fix bug with aggregating around is null values on Django 3.1. Fix is null returning None for missing fields in JsonFields.
3.0.1	2020-11-12	Add get_* functions for the ddb_* admin options. Add length function to string fields. Add support for DB query explain via .explain url. Prevent exception when getting SQL view of pure aggregates. Fix incorrect handling of ISO dates whose day portion is less than 13. Python 3.9 support.
3.0.0	2020-11-09	The format of ddb_default_filters has changed. Path and prettyPath have been removed from fields and filters on JSON responses. Choice and is null fields use human readable values in filters. Choice fields have a raw sub field for accessing the underlying values. Starts with, regex, etc have been removed form choice fields, equivalents are on raw. Verbose_names and short_descriptions are used for display in the web frontend and CSV. Equals and not equals for JSON and arrays. JSON field filter supports lists and objects. Array values are now JSON encoded across the board. Backfill saved views for above changes to filter formats. Pickup calculated fields on inlines when there is no actual admin. Fix bug where ID’s and annotations on inlines were visible to users without perms. Support for aggregation and functions on annotated fields. Annotations now respect ddb_hide. Admin links to the Data Browser respect ddb_ignore.
2.2.21	2020-11-02	Reject ambiguous date and datetime values in filters.
2.2.20	2020-10-22	Fix bug with ArrayField on Django>=3.0
2.2.19	2020-10-19	Support for annotations on inlines.
2.2.18	2020-10-18	Support for profiling CSV etc output. See CONTRIBUTING.rst Performance improvements for large result sets.
2.2.17	2020-10-15	Performance improvements for large result sets. Fix error when choices field has an unexpected value.
2.2.16	2020-09-28	Fix being unable to reorder aggregates when there is no pivot. Fix back button sometimes not remembering column reorderings. Fix reordering columns while a long reload is in progress causes an error.
2.2.15	2020-09-27	Handle callables in ModelAdmin.list_display. Add data_browser.helpers.attributes. Deprecated @ddb_hide in favor of @attributes(ddb_hide=True). Render safestrings returned by calculated fields as HTML. Respect the boolean attribute on calculated fields. Aside from declared booleans, calculated fields now always format as strings.
2.2.14	2020-09-20	Saved view style tweaks. Only reload on field delete when it might change the results. Add UI controls for reordering fields.
2.2.13	2020-09-13	Add .sql format to show raw SQL query. Min and max for date and datetime fields. Add ddb_default_filters. Integrated cProfile support via .profile and .pstats.
2.2.12	2020-09-09	DurationField support. Sort newly added date (etc) fields by default. Fix JSONField support when psycopg2 is not installed. Fix bug with number formatting and pivoted data. Fix error with multiple non adjacent filters on the same field. Fix error with naive DateTimeFields.
2.2.11	2020-08-31	Minor enhancements and some small fixes.
2.2.10	2020-08-31	Minor enhancements.
2.2.9	2020-08-25	Small fixes.
2.2.8	2020-08-23	Small fixes.
2.2.7	2020-08-22	Small fixes.
2.2.6	2020-08-16	Basic JSONField support.
2.2.5	2020-08-01	Bug fix.
2.2.4	2020-08-01	Additional field support. Minor features and bug fixes.
2.2.3	2020-07-31	File and Image field support
2.2.2	2020-07-26	Better support for choice fields.
2.2.1	2020-07-25	Performance tweaks.
2.2.0	2020-07-21	Sort and filter annotated fields.
2.1.2	2020-07-11	Minor bug fixes.
2.1.1	2020-07-06	Bug fixes. The representation of empty pivot cells has changed in the JSON.
2.1.0	2020-07-06	Bring views into the JS frontend. Implement row limits on results. All existing saved views will be limited to 1000 rows. Better loading and error status indication. Lock column headers.
2.0.5	2020-06-20	Bug fixes.
2.0.4	2020-06-18	Fix Py3.6 support.
2.0.3	2020-06-14	Improve filtering on aggregates when pivoted.
2.0.2	2020-06-14	Improve fonts and symbols.
2.0.1	2020-06-14	Improve sorting when pivoted.
2.0.0	2020-06-14	Pivot tables. All public view URL’s have changed. The JSON data format has changed.
1.2.6	2020-06-08	Bug fixes.
1.2.5	2020-06-08	Bug fixes.
1.2.4	2020-06-03	Calculated fields interact better with aggregation.
1.2.3	2020-06-02	JS error handling tweaks.
1.2.2	2020-06-01	Minor fix.
1.2.1	2020-05-31	Improved date handling.
1.2.0	2020-05-31	Support for date functions “year”, “month” etc and filtering based on “now”.
1.1.6	2020-05-24	Stronger sanitizing of URL strings.
1.1.5	2020-05-23	Fix bug aggregating time fields.
1.1.4	2020-05-23	Fix breaking bug with GenericInlineModelAdmin.
1.1.3	2020-05-23	Cosmetic fixes.
1.1.2	2020-05-22	Cosmetic fixes.
1.1.1	2020-05-20	Cosmetic fixes.
1.1.0	2020-05-20	Aggregate support.
1.0.2	2020-05-17	Py3.6 support.
1.0.1	2020-05-17	Small fixes.
1.0.0	2020-05-17	Initial version.

Project details

These details have not been verified by PyPI

Project links

Homepage

Release history Release notifications | RSS feed

4.2.10

Apr 9, 2024

4.2.9

Apr 2, 2024

4.2.8

Feb 11, 2024

4.2.7

Feb 4, 2024

4.2.6

Feb 4, 2024

4.2.5

Jul 20, 2023

4.2.4

Jul 2, 2023

4.2.3

Jun 15, 2023

4.2.2

Jun 8, 2023

4.2.1

May 21, 2023

4.1.1

May 7, 2023

4.1.0

May 1, 2023

4.0.17

Apr 25, 2023

4.0.16

Apr 16, 2023

4.0.15

Apr 11, 2023

4.0.14

Apr 5, 2023

4.0.13

Mar 6, 2023

4.0.12

May 3, 2022

4.0.11

Apr 12, 2022

4.0.10

Apr 12, 2022

4.0.9

Jan 4, 2022

4.0.8

Dec 12, 2021

4.0.7

Aug 16, 2021

4.0.6

Aug 10, 2021

4.0.5

Aug 9, 2021

4.0.4

Jul 4, 2021

4.0.3

Jun 18, 2021

4.0.2

Apr 12, 2021

4.0.1

Apr 11, 2021

4.0.0

Mar 13, 2021

3.3.0

Feb 19, 2021

This version

3.2.5

Feb 7, 2021

3.2.4

Feb 2, 2021

3.2.3

Jan 11, 2021

3.2.2

Dec 30, 2020

3.2.1

Dec 30, 2020

3.2.0

Dec 30, 2020

3.1.4

Dec 19, 2020

3.1.3

Dec 13, 2020

3.1.2

Dec 9, 2020

3.1.1

Dec 1, 2020

3.1.0

Nov 29, 2020

3.0.4

Nov 28, 2020

3.0.3

Nov 22, 2020

3.0.2

Nov 18, 2020

3.0.1

Nov 12, 2020

3.0.0

Nov 9, 2020

2.2.21

Nov 2, 2020

2.2.20

Oct 22, 2020

2.2.19

Oct 19, 2020

2.2.18

Oct 18, 2020

2.2.17

Oct 15, 2020

2.2.16

Sep 28, 2020

2.2.15

Sep 27, 2020

2.2.14

Sep 20, 2020

2.2.13

Sep 13, 2020

2.2.12

Sep 9, 2020

2.2.11

Sep 6, 2020

2.2.10

Aug 31, 2020

2.2.9

Aug 25, 2020

2.2.8

Aug 23, 2020

2.2.7

Aug 22, 2020

2.2.6

Aug 16, 2020

2.2.5

Aug 1, 2020

2.2.4

Aug 1, 2020

2.2.3

Jul 31, 2020

2.2.2

Jul 26, 2020

2.2.1

Jul 25, 2020

2.2.0

Jul 21, 2020

2.1.2

Jul 11, 2020

2.1.1

Jul 6, 2020

2.1.0

Jul 6, 2020

2.0.5

Jun 20, 2020

2.0.4

Jun 18, 2020

2.0.3

Jun 15, 2020

2.0.2

Jun 14, 2020

2.0.1

Jun 14, 2020

2.0.0

Jun 14, 2020

1.2.6

Jun 8, 2020

1.2.5

Jun 8, 2020

1.2.4

Jun 3, 2020

1.2.3

Jun 2, 2020

1.2.2

Jun 1, 2020

1.2.1

May 31, 2020

1.2.0

May 31, 2020

1.1.6

May 24, 2020

1.1.5

May 23, 2020

1.1.4

May 23, 2020

1.1.3

May 23, 2020

1.1.2

May 22, 2020

1.1.1

May 20, 2020

1.1.0

May 20, 2020

1.0.2

May 17, 2020

1.0.1

May 17, 2020

1.0.0

May 17, 2020

0.0.9

May 10, 2020

0.0.8

May 9, 2020

0.0.7

May 9, 2020

0.0.6

May 9, 2020

0.0.5

May 9, 2020

0.0.4

May 9, 2020

0.0.3

Apr 25, 2020

0.0.2

Apr 22, 2020

0.0.1

Apr 14, 2020

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

django-data-browser-3.2.5.tar.gz (712.9 kB view details)

Uploaded Feb 7, 2021 Source

Built Distribution

django_data_browser-3.2.5-py3-none-any.whl (435.9 kB view details)

Uploaded Feb 7, 2021 Python 3

File details

Details for the file django-data-browser-3.2.5.tar.gz.

File metadata

Download URL: django-data-browser-3.2.5.tar.gz
Upload date: Feb 7, 2021
Size: 712.9 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/3.3.0 pkginfo/1.6.1 requests/2.25.0 setuptools/53.0.0 requests-toolbelt/0.9.1 tqdm/4.54.1 CPython/3.8.6

File hashes

Hashes for django-data-browser-3.2.5.tar.gz
Algorithm	Hash digest
SHA256	`8be480e788b03cc30884b7ba4c0565372b93ec558a2e4f684a6280d628e196ee`
MD5	`2837923e43442b52fdbcbf71d6f2b232`
BLAKE2b-256	`04e2d5607c91acfe01530fa556af85362412ec2019f27cd4677dd38ee147ea3a`

See more details on using hashes here.

File details

Details for the file django_data_browser-3.2.5-py3-none-any.whl.

File metadata

Download URL: django_data_browser-3.2.5-py3-none-any.whl
Upload date: Feb 7, 2021
Size: 435.9 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: twine/3.3.0 pkginfo/1.6.1 requests/2.25.0 setuptools/53.0.0 requests-toolbelt/0.9.1 tqdm/4.54.1 CPython/3.8.6

File hashes

Hashes for django_data_browser-3.2.5-py3-none-any.whl
Algorithm	Hash digest
SHA256	`587294b1f81267106405e3df6b011db856ac785fe7fda6710f59493569c31e93`
MD5	`ace64c6a1d34f9b487144aed3ac2a530`
BLAKE2b-256	`6664096a416c4ff4e5e1908a577ee0b80d28ace10441d60a94d8fcf5876bdc13`