quak /kwæk/

an anywidget for data that talks like a duck 🦆

quak is a scalable data profiler for quickly scanning large tables, capturing interactions as executable SQL queries.

• interactive 🖱️ mouse over column summaries, cross-filter, sort, and slice rows.
• fast ⚡ built with Mosaic; views are expressed as SQL queries lazily executed by DuckDB.
• flexible 🔄 supports many data types and formats via Apache Arrow and the dataframe interchange protocol.
• reproducible 📓 a UI for building complex SQL queries; materialize views in the kernel for further analysis.

install

[!WARNING] quak is a prototype exploring a high-performance data profiler based on anywidget. It is not production-ready. Expect bugs. Open-sourced for SciPy 2024.

pip install quak

usage

The easiest way to get started with quak is using the IPython cell magic.

%load_ext quak

import polars as pl

df = pl.read_parquet("https://github.com/uwdata/mosaic/raw/main/data/athletes.parquet")
df

quak hooks into Jupyter's display mechanism to automatically render any dataframe-like object (implementing the Python dataframe interchange protocol) using quak.Widget instead of the default display.

Alternatively, you can use quak.Widget directly:

import polars as pl
import quak

df = pl.read_csv("https://github.com/uwdata/mosaic/raw/main/data/athletes.parquet")
widget = quak.Widget(df)
widget

interacting with the data

quak is a UI for quickly scanning and exploring large tables. However, it is more than that. A side effect of quak's Mosaic-based architecture is that it captures all user interactions as SQL queries.

At any point, table state can be accessed as a query,

widget.sql # SELECT * FROM df WHERE ...

which for convenience can be executed in the kernel to materialize the view for further analysis:

widget.data() # returns duckdb.DuckDBPyRelation object

By representing UI state as SQL, quak makes it easy to generate complex queries via interactions that would be challenging to write manually, while keeping them reproducible.

contributing

Contributors welcome! Check the Contributors Guide to get started. Note: I'm wrapping up my PhD, so I might be slow to respond. Please open an issue before contributing a new feature.

references

quak pieces together many important ideas from the web and Python data science ecosystems. It serves as an example of what you can achieve by embracing these platforms for their strengths.

• Observable's data table: Inspiration for the UI design and user interactions.
• Mosaic: The foundation for linking databases and interactive table views.
• Apache Arrow: Support for various data types and efficient data interchange between JS/Python.
• DuckDB: An amazingly engineered piece of software that makes SQL go vroom.