A simple benchmarking tool
Project description
Simple performance measurement tool
See the documentation for more details.
Installation
neurtu requires Python 2.7 or 3.4+, it can be installed with,
pip install neurtu
pandas is an optional (but highly recommended) dependency.
Quickstart
Single benchmark
Measuring the run time,
>>> from neurtu import timeit, delayed >>> timeit(delayed(sum)(range(100000)), repeat=3) {'wall_time_max': 0.0018, 'wall_time_mean': 0.0017, 'wall_time_min': 0.0015, 'wall_time_std': 0.00011}
which will internally call timeit.Timer. Similarly to IPython’s %timeit, the number of runs will be determined at runtime to mitigate the finite resolution of the timer (on Windows it’s 16 ms!). In addition, each evaluation will be here repeated 3 times (default) to measure statistics.
Similarly, the memory use can be measured with,
>>> from neurtu import memit, delayed >>> memit(delayed(sorted)(list(range(100000)))) {'peak_memory_max': 0.765, 'peak_memory_mean': 0.757, 'peak_memory_min': 0.753, 'peak_memory_std': 0.00552}
Generic benchmarks
Both timeit and memit are aliases for the Benchmark class which can be used with one or several metrics,
from neurtu import Benchmark, delayed Benchmark(wall_time=True, peak_memory=True)( delayed(sorted)(range(100000)), number=3)
Currently supported metrics are wall_time, peak_memory as well as cpu_time (Linux and Mac OS only).
Parametric benchmarks
The timeit, memit and Benchmark also accept as input sequence of delayed objects, tagged with the tags parameter,
>>> from neurtu import timeit, delayed
>>> timeit(delayed(sorted, tags={'N': N})(range(N))
... for N in [1000, 10000, 100000])
N wall_time_max wall_time_mean wall_time_min wall_time_std
0 1000 0.000024 0.000024 0.000024 5.951794e-08
1 10000 0.000319 0.000319 0.000318 4.723035e-07
2 100000 0.003695 0.003686 0.003678 7.144085e-06
which will produce a pandas.DataFrame with the measures if pandas is installed and a list of dictionaries otherwise.
In general, we can pass any iterable to the benchmark functions. For instance the above example is equivalent to,
>>> from neurtu import timeit, delayed
>>> def delayed_cases():
... for N in [1000, 10000, 100000]:
... yield delayed(sorted, tags={'N': N})(range(N))
>>> timeit(delayed_cases())
Delayed evaluation
The delayed function is a partial implementation of the dask.delayed API. It models operations as a chained list of delayed objects that are not evaluated until the compute() method is called.
>>> from neurtu import delayed
>>> x = delayed('some string').split(' ')[::-1]
>>> x
<Delayed('some string').split(' ')[slice(None, None, -1)]>
>>> x.compute()
['string', 'some']
Attribute access, indexing as well as function and method calls are supported. Left function composition (e.g. func(delayed(obj))) and binary operations (e.g. delayed(op) + 1) are currently not supported, neither is the composition of multiple delayed objects, use dask.delayed for those.
Scientific computing usage
A typical use case, occurs when manipulating objects with a scikit-learn API,
res = Benchmark(wall_time=True, cpu_time=True)(
delayed(NearestNeighbors, tags={'n_jobs': n_jobs})(n_jobs=n_jobs).fit(X)
for n_jobs in range(1, 10))
Motivation
The API was strongly inspired by joblib.Parallel.
The package name was taken from the Basque word neurtu meaning “to measure / evaluate”.
License
neurtu is released under the 3-clause BSD license.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for neurtu-0.1a2-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6547737e160945034d7a1bcf3b0a8fb603c5e96efeabbc943c32bcb203c35deb |
|
MD5 | 77991f2ac24c103c85f33261c02cb297 |
|
BLAKE2b-256 | 0d1c08e7f025aab9997aa4015d1e10f231b4e78a204966479edf2a83cf22de01 |