quak /kwæk/

an anywidget for data that talks like a duck

quak is a scalable data profiler for quickly scanning large tables, capturing interactions as executable SQL queries.

• interactive 🖱️ mouse over column summaries, cross-filter, sort, and slice rows.
• fast ⚡ built with Mosaic; views are expressed as SQL queries lazily executed by DuckDB.
• flexible 🔄 supports many data types and formats via Apache Arrow, the dataframe interchange protocol, and the Arrow PyCapsule Interface.
• reproducible 📓 a UI for building complex SQL queries; materialize views in the kernel for further analysis.

install

[!WARNING] quak is a prototype exploring a high-performance data profiler based on anywidget. It is not production-ready. Expect bugs. Open-sourced for SciPy 2024.

pip install quak

usage

The easiest way to get started with quak is using the IPython cell magic.

%load_ext quak

import polars as pl

df = pl.read_parquet("https://github.com/uwdata/mosaic/raw/main/data/athletes.parquet")
df

quak hooks into Jupyter's display mechanism to automatically render any dataframe-like object (implementing the Python dataframe interchange protocol or Arrow PyCapsule Interface) using quak.Widget instead of the default display.

Alternatively, you can use quak.Widget directly:

import polars as pl
import quak

df = pl.read_parquet("https://github.com/uwdata/mosaic/raw/main/data/athletes.parquet")
widget = quak.Widget(df)
widget

interacting with the data

quak captures all user interactions as queries.

At any point, table state can be accessed as SQL,

widget.sql # SELECT * FROM df WHERE ...

which for convenience can be executed in the kernel to materialize the view for further analysis:

widget.data() # returns duckdb.DuckDBPyRelation object

By representing UI state as SQL, quak makes it easy to generate complex queries via interactions that would be challenging to write manually, while keeping them reproducible.

using quak in marimo

quak can also be used in marimo notebooks, which provide out-of-the-box support for anywidget:

import marimo as mo
import polars as pl
import quak

df = pl.read_parquet("https://github.com/uwdata/mosaic/raw/main/data/athletes.parquet")
widget = mo.ui.anywidget(quak.Widget(df))
widget

contributing

Contributors welcome! Check the Contributors Guide to get started. Note: I'm wrapping up my PhD, so I might be slow to respond. Please open an issue before contributing a new feature.

references

quak pieces together many important ideas from the web and Python data science ecosystems. It serves as an example of what you can achieve by embracing these platforms for their strengths.

• Observable's data table: Inspiration for the UI design and user interactions.
• Mosaic: The foundation for linking databases and interactive table views.
• Apache Arrow: Support for various data types and efficient data interchange between JS/Python.
• DuckDB: An amazingly engineered piece of software that makes SQL go vroom.