redash-stmo 2020.5.1

Build and update the local Docker image

Many of the helpers below will implicitely run docker-compose to start the Redash containers and in effect automatically build the local Docker image as well if it doesn’t exist.

If you’d like to build the local Docker image separately or if you’d like to fetch the latest version of the base Redash Docker image (or its child image of the Mozilla Redash fork), e.g. in the event of a new rebase by Mozilla staff, please run the following:

make build

Behind the scenes

This will run docker-compose build --pull which will pull updates to the Docker images used by the docker-compose setup, including the Redash, Redis and Postgres images.

Create the database

On you command line run this ONCE to create the database for Redash/redash-stmo setup:

make database

This uses Redash’s own ability and redash-stmo is just set up to reuse it.

Behind the scenes

This will run docker-compose to create the server container that is running the Redash Python server and in effect the Redis and Postgres containers, too.

It will then initialize the Postgres tables needed for Redash.

Install npm modules

Then we’ll install the Redash npm modules inside the server container:

make node_modules

Behind the scenes

This will run npm install inside the server in the /app directory, which is the directory with Redash’s code from the Redash Docker base image.

NOTE, the redash-stmo development setup mounts the /app/node_modules directory as a separate Docker volume, that will be maintained by Docker and won’t show up in or transfer to the host machine where Docker is running.

Start the containers

To start the whole set of Docker containers for a working environment (Redash server, RQ workers, Redis, Postgres) all you need to run is this:

make up

Behind the scenes

This is pretty simply running docker-compose up, to launch all containers of the redash-stmo Docker setup.

NOTE: This requires first installing npm modules inside the container above and creating the database as well!

Run webpack devserver

If you’re developing a Redash extension that includes an additional webpack bundle (which will need to be included in Redash’s webpack build process to be shipped in the client application bundle) you’ll want to use the webpack development server.

It automatically compiles the Redash client application bundle on files changes and proxies requests for the Redash server via a proxy running on port 8080 (instead of the usual Redash port of 5000).

After starting the containers using the description in the above step, open a second terminal and additionally run:

make devserver

Behind the scenes

This will run the webpack devserver in another instance of the server container (not the same as when running make up) and runs a script that listens for files changes to .js and .jsx files in the /extension directory.

When changes are detected, it’ll automatically run Redash’s bundle-extensions script that does the heavy lifting of copying the changed extension files into the /app/client/app/extensions directory, which triggers the webpack devserver to recompile the client application bundle.

NOTE: This requires opening the Redash instance via http://localhost:8080/ instead of http://localhost:5000/ to go through the webpack devserver.

Start shell

In case you need to do any debugging or file system checks inside the server container, you can create a bash shell by running:

make bash

Behind the scenes

Any changes you make here outside the /extension directory (which is mounted as a Docker volume with the current working directory on the Docker host machine) and the following directores are not persisted.

List of directories inside the container that are mounted as Docker volumes:

/extension

Maps the current working directory (where this README.rst is located) on the host machine for developing the extension.

/home/redash/.cache

Used by pip and other scripts,

/app/client/dist

Directory to retain webpack build results, so webpack builds don’t take as long on consecutive runs.

/home/redash/.local

Directory for “user-installed” Python packages. If you’d like you can easily install additonal Python packages with the Docker container user Redash using pip install --user <package>. Installed scripts from those packages will be found under /home/redash/.local/bin but are also automatically added to PATH.

/app/node_modules

Directory for npm modules, that are installed when running npm install inside of /app in the container. Retained to make use of native npm caching between consecutive runs.

Run tests

Running the Python based tests requires first creating a separate database (implemented by the test_database Make task) and then running the test runner inside the container. The test database is not the same as the databse in use for regular development (e.g. to not overwrite development data).

Frontend or integration tests are currently not supported.

To run the tests (from the host machine) run:

make test

This will automatically run the test_database Make task before running the tests.

Behind the scenes

When launching the tests runner it’ll the regular server container, but also set the REDASH_DATABASE_URL environment variable to the test database to prevent overwriting any data that you added to the database the regular Redash interface (e.g. data sources, queries etc).

By default it uses pytest to run the Python tests in /extension, with a number of parameters as defined in the pytest.ini.

If you’d like to add additional parameters to pytest simply appened the command line arguments in pytest.ini.

Alternatively, e.g. if you’d like to use pdb to debug a test, do this:

create the test database from the host machine

make test_database

start a Bash shell in the container

make bash

set the REDASH_DATABASE_URL env var in the container

export REDASH_DATABASE_URL="postgresql://postgres@postgres/tests"

change direcotry to extensio code

cd /extension

run the tests with whatever parameter

pytest -vvv --pdb