Package for calculating and visualising confidence intervals, e.g. for A/B test analysis.
Project description
Spotify Confidence
Python library for AB test analysis.
Why use Spotify Confidence?
Spotify Confidence provides convinience wrappers around statsmodel's various functions for computing p-values and confidence intervalls. With Spotify Confidence it's easy to compute several p-values and confidence bounds in one go, e.g. one for each country or for each date. Each function comes in two versions:
- one that return a pandas dataframe,
- one that returns a Chartify chart.
Spotify Confidence has support calculating p-values and confidence intervals using Z-statistics, Student's T-statistics (or more exactly Welch's T-test), as well as Chi-squared statistics. It also supports a variance reduction technique based on using pre-exposure data to fit a linear model.
There is also a Bayesian alternative in the BetaBinomial class.
Basic Example
import spotify_confidence as confidence
import pandas as pd
data = pd.DataFrame(
{'variation_name': ['treatment1', 'control', 'treatment2', 'treatment3'],
'success': [50, 40, 10, 20],
'total': [100, 100, 50, 60]
}
)
test = confidence.ZTest(
data,
numerator_column='success',
numerator_sum_squares_column=None,
denominator_column='total',
categorical_group_columns='variation_name',
correction_method='bonferroni')
test.summary()
test.difference(level_1='control', level_2='treatment1')
test.multiple_difference(level='control', level_as_reference=True)
test.summary_plot().show()
test.difference_plot(level_1='control', level_2='treatment1').show()
test.multiple_difference_plot(level='control', level_as_reference=True).show()
There's a lot more you can do:
- Segment results by one or more dimensions
- Use non-inferiority margins
- Group sequential tests
- Sample size and power calculations
- etc
See jupyter notebooks in examples folder for more complete examples.
Installation
Spotify Confidence can be installed via pip:
pip install spotify-confidence
Find the latest release version here
Code of Conduct
This project adheres to the Open Code of Conduct By participating, you are expected to honor this code.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file spotify-confidence-3.0.1.tar.gz.
File metadata
- Download URL: spotify-confidence-3.0.1.tar.gz
- Upload date:
- Size: 92.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.11.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 0417d4ca4e8966f7bb58b818d83a2b2e46c95235ff79e61efee9b7f11c74156a |
|
| MD5 | e986d6b8b3c5f4d504529ee421c45b3f |
|
| BLAKE2b-256 | 148ae9f214a2028fbf5bc32c1a425af5b90172d94f2458da43adb97c8c27acd3 |
File details
Details for the file spotify_confidence-3.0.1-py3-none-any.whl.
File metadata
- Download URL: spotify_confidence-3.0.1-py3-none-any.whl
- Upload date:
- Size: 85.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.11.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 5bdc4d7b225a4d1dd427a8afc58cc861ae996497d9fb83aec64c0f5ceefa6cf4 |
|
| MD5 | bc99cd4fcb462310acd7afab7732d716 |
|
| BLAKE2b-256 | 1d87c40160ace7c9239db8645d9103692d7add38adb1211d78fe54d9fabddaa5 |
