Mosaic DuckDB Server

A Python-based server that runs a local DuckDB instance and support queries over Web Sockets or HTTP, returning data in either Apache Arrow or JSON format.

Note: This package provides a local DuckDB server. To instead use DuckDB-WASM in the browser, use the wasmConnector in the mosaic-core package.

Installation and usage

We recommend running the server in an isolated environment with pipx. For example, to directly run the server, use:

pipx run duckdb-server

Alternatively, you can install the server with pip install duckdb-server. Then you can start the server with duckdb-server.

Developer Setup

We use hatch to manage our development setup.

Start the server in development with hatch run serve. The server restarts when you change the code.

To run the tests, use hatch run test:cov.

To set up a local certificate for SSL, use https://github.com/FiloSottile/mkcert.

API

The server supports queries via HTTP GET and POST, and WebSockets. The GET endpoint is useful for debugging. For example, you can query it with this url.

Each endpoint takes a JSON object with a command in the type. The server supports the following commands.

exec

Executes the SQL query in the sql field.

arrow

Executes the SQL query in the sql field and returns the result in Apache Arrow format.

json

Executes the SQL query in the sql field and returns the result in JSON format.

create-bundle

Caches the results of the SQL queries in the queries field and the required datasets.

load-bundle

Loads the bundled results.

Publishing

Run the build with hatch build. Then publish with hatch publish. We publish using tokens so when asked, set the username to __token__ and then use your token as the password. Alternatively, create a .pypirc file.