Skip to main content

Statistical profiling for Python

Project description

https://travis-ci.org/smarkets/statprof.svg?branch=master https://img.shields.io/pypi/v/statprof-smarkets.svg

This package provides a simple statistical profiler for Python.

Python’s default profiler has been lsprof for several years. This is an instrumenting profiler, which means that it saves data on every action of interest. In the case of lsprof, it runs at function entry and exit. This has problems: it can be expensive due to frequent sampling, and it is blind to hot spots within a function.

In contrast, statprof samples the call stack periodically (by default, 1000 times per second), and it correctly tracks line numbers inside a function. This means that if you have a 50-line function that contains two hot loops, statprof is likely to report them both accurately.

How to get it

Use pip!

pip install statprof-smarkets

Warning: it uses statprof as Python module name so this will conflict with original statprof installation if present.

GitHub project page: https://github.com/smarkets/statprof

PyPI page: https://pypi-hypernode.com/pypi/statprof-smarkets

Basic usage

It’s easy to get started with statprof:

import statprof

statprof.start()
try:
    my_questionable_function()
finally:
    statprof.stop()
    statprof.display()

Or with a contextmanager:

import statprof

with statprof.profile():
    my_questionable_function()

Or from command line:

$ python -m statprof script.py
# or
$ python -m statprof -m script
# or (this may depend on bash because http://www.gnu.org/software/bash/manual/bashref.html#ANSI_002dC-Quoting)
$ python -m statprof -c "import hashlib"$'\n'"for i in range(10000): hashlib.md5(str(i)).hexdigest()"

For more comprehensive help, run pydoc statprof.

Portability

Because statprof uses the Unix itimer signal facility, it does not currently work on Windows. (Patches to improve portability would be most welcome.)

Implementation notes

The statprof profiler works by setting the Unix profiling signal ITIMER_PROF to go off after the interval you define in the call to reset(). When the signal fires, a sampling routine is run which looks at the current procedure that’s executing, and then crawls up the stack, and for each frame encountered, increments that frame’s code object’s sample count. Note that if a procedure is encountered multiple times on a given stack, it is only counted once. After the sampling is complete, the profiler resets profiling timer to fire again after the appropriate interval.

Meanwhile, the profiler keeps track, via os.times(), how much CPU time (system and user – which is also what ITIMER_PROF tracks), has elapsed while code has been executing within a start()/stop() block.

The profiler also tries (as much as possible) to avoid counting or timing its own code.

Changelog

0.2.0

  • forked

  • refactored

  • added configurable display format (displays full paths by default now)

  • ability to run whole scripts under statprof from command line (thanks to Vincent Driessen and Antony Lee

  • added support for python -mstatprof -c cmd invocation (thanks to Antony Lee)

History

This package was originally written and released by Andy Wingo. It was ported to modern Python by Alex Frazer, and posted to GitHub by Jeff Muizelaar. Maintained by Bryan O’Sullivan, was forked by Smarkets due to package not being maintaned anymore.

Reporting bugs, contributing patches

Please report bugs using the GitHub issue tracker.

If you’d like to contribute patches, please do - the source is on GitHub, so please just issue a pull request.

$ git clone git://github.com/smarkets/statprof

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

statprof-smarkets-1.0.0.tar.gz (18.4 kB view details)

Uploaded Source

Built Distribution

statprof-smarkets-1.0.0.linux-x86_64.tar.gz (13.4 kB view details)

Uploaded Source

File details

Details for the file statprof-smarkets-1.0.0.tar.gz.

File metadata

File hashes

Hashes for statprof-smarkets-1.0.0.tar.gz
Algorithm Hash digest
SHA256 b1ecadfb86ead5c75d987fbc8cc0f53e1a3a6c6ea644918a67b3e8bb5036372f
MD5 723d9c727da43a342042fe00486857d9
BLAKE2b-256 912d47e9c0ec4093242c2fb3b216c2379d2ef29df92275ebc234c2f8bd148eef

See more details on using hashes here.

File details

Details for the file statprof-smarkets-1.0.0.linux-x86_64.tar.gz.

File metadata

File hashes

Hashes for statprof-smarkets-1.0.0.linux-x86_64.tar.gz
Algorithm Hash digest
SHA256 a332353bd2ea39620e0ee9f9b1999cb382800a7ad1edc66eea23bac9b24e3699
MD5 04da8b484e805cbf83b02006cac3e814
BLAKE2b-256 d1e0dfbd5c4915ed01829e932a1b8706b4989ceaf4e499a6c0c0ff430c88e198

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page