Flexible and fast sampling in Python
Project description
BlackJAX
What is BlackJAX?
BlackJAX is a library of samplers for JAX that works on CPU as well as GPU.
It is not a probabilistic programming library. However it integrates really well with PPLs as long as they can provide a (potentially unnormalized) log-probability density function compatible with JAX.
Who should use BlackJAX?
BlackJAX should appeal to those who:
- Have a logpdf and just need a sampler;
- Need more than a general-purpose sampler;
- Want to sample on GPU;
- Want to build upon robust elementary blocks for their research;
- Are building a probabilistic programming language;
- Want to learn how sampling algorithms work.
Quickstart
Installation
You can install BlackJAX using pip:
pip install blackjax
or via conda-forge:
conda install -c conda-forge blackjax
Nightly builds (bleeding edge) of Blackjax can also be installed using pip:
pip install blackjax-nightly
BlackJAX is written in pure Python but depends on XLA via JAX. By default, the version of JAX that will be installed along with BlackJAX will make your code run on CPU only. If you want to use BlackJAX on GPU/TPU we recommend you follow these instructions to install JAX with the relevant hardware acceleration support.
Example
Let us look at a simple self-contained example sampling with NUTS:
import jax
import jax.numpy as jnp
import jax.scipy.stats as stats
import numpy as np
import blackjax
observed = np.random.normal(10, 20, size=1_000)
def logdensity_fn(x):
logpdf = stats.norm.logpdf(observed, x["loc"], x["scale"])
return jnp.sum(logpdf)
# Build the kernel
step_size = 1e-3
inverse_mass_matrix = jnp.array([1., 1.])
nuts = blackjax.nuts(logdensity_fn, step_size, inverse_mass_matrix)
# Initialize the state
initial_position = {"loc": 1., "scale": 2.}
state = nuts.init(initial_position)
# Iterate
rng_key = jax.random.PRNGKey(0)
for _ in range(100):
rng_key, nuts_key = jax.random.split(rng_key)
state, _ = nuts.step(nuts_key, state)
See the documentation for more examples of how to use the library: how to write inference loops for one or several chains, how to use the Stan warmup, etc.
Philosophy
What is BlackJAX?
BlackJAX bridges the gap between "one liner" frameworks and modular, customizable libraries.
Users can import the library and interact with robust, well-tested and performant samplers with a few lines of code. These samplers are aimed at PPL developers, or people who have a logpdf and just need a sampler that works.
But the true strength of BlackJAX lies in its internals and how they can be used to experiment quickly on existing or new sampling schemes. This lower level exposes the building blocks of inference algorithms: integrators, proposal, momentum generators, etc and makes it easy to combine them to build new algorithms. It provides an opportunity to accelerate research on sampling algorithms by providing robust, performant and reusable code.
Why BlackJAX?
Sampling algorithms are too often integrated into PPLs and not decoupled from the rest of the framework, making them hard to use for people who do not need the modeling language to build their logpdf. Their implementation is most of the time monolithic and it is impossible to reuse parts of the algorithm to build custom kernels. BlackJAX solves both problems.
How does it work?
BlackJAX allows to build arbitrarily complex algorithms because it is built around a very general pattern. Everything that takes a state and returns a state is a transition kernel, and is implemented as:
new_state, info = kernel(rng_key, state)
kernels are stateless functions and all follow the same API; state and information related to the transition are returned separately. They can thus be easily composed and exchanged. We specialize these kernels by closure instead of passing parameters.
Contributions
What contributions?
We value the following contributions:
- Bug fixes
- Documentation
- High-level sampling algorithms from any family of algorithms: random walk, hamiltonian monte carlo, sequential monte carlo, variational inference, inference compilation, etc.
- New building blocks, e.g. new metrics for HMC, integrators, etc.
How to contribute?
- Run
pip install -r requirements.txtto install all the dev dependencies. - Run
pre-commit run --all-filesandmake testbefore pushing on the repo; CI should pass if these pass locally.
Citing Blackjax
To cite this repository:
@software{blackjax2020github,
author = {Lao, Junpeng and Louf, R\'emi},
title = {{B}lackjax: A sampling library for {JAX}},
url = {http://github.com/blackjax-devs/blackjax},
version = {<insert current release tag>},
year = {2020},
}
In the above bibtex entry, names are in alphabetical order, the version number should be the last tag on the main branch.
Acknowledgements
Some details of the NUTS implementation were largely inspired by Numpyro's.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file blackjax-nightly-0.9.6.post170.tar.gz.
File metadata
- Download URL: blackjax-nightly-0.9.6.post170.tar.gz
- Upload date:
- Size: 286.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.9.17
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 53d7b23558e1b595e8d08e850149ef00d164e830b9d8dab16b4d60f7c1efad32 |
|
| MD5 | 6c0fb5884cba3a5097ba18db80d1e919 |
|
| BLAKE2b-256 | 2dbc59be9cfe205572292dd3c3e3f55de68ef05bdd21f9e7e5cc07c2d5599738 |
File details
Details for the file blackjax_nightly-0.9.6.post170-py3-none-any.whl.
File metadata
- Download URL: blackjax_nightly-0.9.6.post170-py3-none-any.whl
- Upload date:
- Size: 300.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.9.17
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | c9c84c30bc950aa198314e3cd82489c44fbc0d85ff94d8f8408835e6b8635dff |
|
| MD5 | 92a40b9075865db90925525a5fee6cb5 |
|
| BLAKE2b-256 | 5409bfe0dfe78317bd6c2fc0f540339ee6848ca6a3912f58611422da56c7b180 |
