A Django app for interactive user friendly browsing of a Django projects DB.

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for tolomea from gravatar.com tolomea

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Project description

Interactive and user friendly querying of Django project DBs.

screenshot

Features

  • Zero config, if it's in the admin it's in the browser
  • Select fields (including calculated fields), aggregate, filter and sort
  • Automatically follow OneToOneFields and ForeignKeys
  • Respects per user admin permissions
  • Share views by URL
  • Save views and optionally make them available to services like Google sheets
  • Download views as CSV or JSON

Roadmap (in no particular order)

  • ToMany support
  • Advanced filtering
  • PII controls
  • Graphs
  • Pivoting

Demo

There is a live demo site available. The Django project is a small e-commerce site selling microservices.

Source: https://github.com/tolomea/data-browser-demo

Admin: https://data-browser-demo.herokuapp.com/admin/

Data Browser: https://data-browser-demo.herokuapp.com/data-browser/

Because it's hosted on Heroku free tier it might take a while to respond to the first page load.

Installation

  1. Run pip install django-data-browser
  2. Add "data_browser" to installed_apps.
  3. Add path("data-browser/", include("data_browser.urls")) to your urls.
  4. Run python manage.py migrate.

Settings

Name Docs Section Function
DATA_BROWSER_ALLOW_PUBLIC Security Allow saved views to be accessed without admin login in limitied circumstances.
DATA_BROWSER_DEV Development Enable proxying frontend to JS dev server.
DATA_BROWSER_FE_DSN Sentry The DSN the frontend sentry should report to, disabled by default

Security

There are two types of django views in the Data Browser.

Queries support general querying of the database (checked against the users admin permissions) but can only be accessed by Django "staff members".

Views can be accessed by anyone but they can only be used to access a query that has been saved and made public and they have long random URL's.

You can use the admin permission data_browser | view | Can make a saved view publically available to restrict who can make views public. To be public the view must be marked as public and owned by someone who has the permission. Users without the permission can not mark views as public and can not edit any view that is marked public.

Additionally the entire public views system is gated by the Django settings value DATA_BROWSER_ALLOW_PUBLIC.

Sentry

The frontend code has builtin Sentry support, it is disabled by default. To enable it set the Django settings value DATA_BROWSER_FE_DSN, for example to set it to the Data Browser project Sentry use:

DATA_BROWSER_FE_DSN = "https://af64f22b81994a0e93b82a32add8cb2b@o390136.ingest.sentry.io/5231151"

Customization and Performance

For concrete fields (as oppose to calculated ones) the Data Browser will do appropriate select and prefetch related calls to minimise it's database impact.

The Data Browser calls the normal admin get_queryset functions. You can use these to customize querysets as needed.

If necessary you can test to see if the databrowser is making the call as follows:

if request.databrowser:
    # Data Browser specific customization

This is particularly useful if you want to route the Data Browser to a DB replica.

The Data Browser also calls get_fieldsets to find out what fields the current user can access. When it does this it always passes a newly constructed instance of the relevant model. This is necessary to work around Django's User admin messing with the fieldsets when None is passed.

Development

The easiest way to develop this is against your existing client project.

The compiled Javascript is checked into the repo, so if only want to mess with the Python then it's sufficient to:

  1. Install the Data Browser in editable mode pip install -e <directory to your git clone>.

If you want to modify the Javascript then you also need to:

  1. Enable proxying to the JS dev server by adding DATA_BROWSER_DEV = True to your settings.
  2. Run the Javascript dev server with WDS_SOCKET_PORT=3000 PUBLIC_URL=data_browser npm start. The WDS_SOCKET_PORT is so the proxied JS can find it's dev server. The PUBLIC_URL tells the JS dev server what path to serve from and should be the same as the URL you have mounted the Data Browser on in you urls file.

To run the Python tests, in the top level of your git clone run pip install -r requirements.txt then pytest.

There is also pre-commit config for lint etc to enable this run pip install pre-commit && pre-commit install then lint will run on git commit. The linting includes Black and isort autoformatting.

To build the JS, move the files around appropriately and recreate the wheels run build.sh.

During development it can be useful to look at the .ctx and .json views. The .ctx view will show you the initial context being passed to the Javascript on page load. The .json view is the actual API request the Javascript uses to fetch query results.

Structure

structure

Terminology

Term Meaning
aggregate Corresponds to a Django aggregation function.
bound query A query that has been validated against the config.
concrete A field that can be sorted and filtered
config Information that doesn't change based on the particular query, includes all the models and their fields.
field name Just the name of the field e.g. created_time.
field path Includes information on how to reach the model the field is on e.g. ["order","seller","created_time"].
function Corresponds to a Django database function for transforming a value, e.g. ExtractYear.
model name Fullstop separated app and model names e.g. myapp.MyModel, also includes synthetic 'models' for hosting aggregate and function fields.
model path Like field path for the model the field is on.
model In Python the actual model class, in Javascript the model name as above.
pretty... User friendly field, and path values
query The information that changes with the query being done, in the Javascript this also includes the results.
type A data type, like string or number
view A saved query.

Most of the code deals with "models" that have "fields" that have "types". In this context a "model" is just anything which might have fields. So types also have associated models which hold the relevant aggregate and function fields. Fields also include foreign keys which may refer to other models. The special meanings of foreignkeys, aggregates, functions and calculated fields is confined to orm.py

Fields have 4 main properties.

Poperty Meaning and impact
name The only required one.
type If set then this field can be added to a query and will return results of the specified type.
concrete Can this field be sorted and filtered. Requires type to be set.
model If set then this field has additional nested fields that are detailed on the given model.

Release History

Version Date Summary
1.2.6 2020-06-08 Bug fixes
1.2.5 2020-06-08 Bug fixes
1.2.4 2020-06-03 Calculated fields interact better with aggregation.
1.2.3 2020-06-02 JS error handling tweaks
1.2.2 2020-06-01 Minor fix
1.2.1 2020-05-31 Improved date handling
1.2.0 2020-05-31 Support for date functions "year", "month" etc and filtering based on "now"
1.1.6 2020-05-24 Stronger sanitizing of URL strings
1.1.5 2020-05-23 Fix bug aggregating time fields
1.1.4 2020-05-23 Fix breaking bug with GenericInlineModelAdmin
1.1.3 2020-05-23 Cosmetic fixes
1.1.2 2020-05-22 Cosmetic fixes
1.1.1 2020-05-20 Cosmetic fixes
1.1.0 2020-05-20 Aggregate support
1.0.2 2020-05-17 Py3.6 support
1.0.1 2020-05-17 Small fixes
1.0.0 2020-05-17 Initial version

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for tolomea from gravatar.com tolomea

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

4.2.10

4.2.9

4.2.8

4.2.7

4.2.6

4.2.5

4.2.4

4.2.3

4.2.2

4.2.1

4.1.1

4.1.0

4.0.17

4.0.16

4.0.15

4.0.14

4.0.13

4.0.12

4.0.11

4.0.10

4.0.9

4.0.8

4.0.7

4.0.6

4.0.5

4.0.4

4.0.3

4.0.2

4.0.1

4.0.0

3.3.0

3.2.5

3.2.4

3.2.3

3.2.2

3.2.1

3.2.0

3.1.4

3.1.3

3.1.2

3.1.1

3.1.0

3.0.4

3.0.3

3.0.2

3.0.1

3.0.0

2.2.21

2.2.20

2.2.19

2.2.18

2.2.17

2.2.16

2.2.15

2.2.14

2.2.13

2.2.12

2.2.11

2.2.10

2.2.9

2.2.8

2.2.7

2.2.6

2.2.5

2.2.4

2.2.3

2.2.2

2.2.1

2.2.0

2.1.2

2.1.1

2.1.0

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

This version

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.2

1.0.1

1.0.0

0.0.9

0.0.8

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

django-data-browser-1.2.6.tar.gz (313.0 kB view details)

Uploaded Source

Built Distribution

django_data_browser-1.2.6-py3-none-any.whl (3.7 MB view details)

Uploaded Python 3

File details

Details for the file django-data-browser-1.2.6.tar.gz.

File metadata

  • Download URL: django-data-browser-1.2.6.tar.gz
  • Upload date:
  • Size: 313.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/47.1.1 requests-toolbelt/0.9.1 tqdm/4.45.0 CPython/3.8.0

File hashes

Hashes for django-data-browser-1.2.6.tar.gz
Algorithm Hash digest
SHA256 dc3c9e2db1f9898148f4cc65b23eb1d70eeba4e29e93dc0036eb39329473e83c
MD5 b72acfcc0c82032fe73b2f1fde621231
BLAKE2b-256 f804b733229be2946597501e97d88435610a1dac97192a64428f07980001720b

See more details on using hashes here.

File details

Details for the file django_data_browser-1.2.6-py3-none-any.whl.

File metadata

  • Download URL: django_data_browser-1.2.6-py3-none-any.whl
  • Upload date:
  • Size: 3.7 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/47.1.1 requests-toolbelt/0.9.1 tqdm/4.45.0 CPython/3.8.0

File hashes

Hashes for django_data_browser-1.2.6-py3-none-any.whl
Algorithm Hash digest
SHA256 ebde386b9641f649f989a155cc2a80b429e58a85f58b181bf4af819f0976fdb1
MD5 25dca778c16df2ecf9b0cc9ff68a9006
BLAKE2b-256 f32c0fbb8c7e2668a5f97b357395959103c89a225d4b34cf512491caa597a644

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page