Skip to main content

A single-threaded alternative to tqdm.

Project description

GithubActions ReadTheDocs Pypi Downloads Codecov

ProgIter

ProgIter lets you measure and print the progress of an iterative process. This can be done either via an iterable interface or using the manual API. Using the iterable interface is most common.

https://i.imgur.com/Jqfg8w0.gif

Basic usage is:

>>> from progiter import ProgIter
>>> for n in ProgIter(range(1000)):
>>>     ...  # do some work

The basic ProgIter is unthreaded. This differentiates it from tqdm and rich.progress which use a threaded implementation. The choice of implementation has different tradeoffs and neither is strictly better than the other. An unthreaded progress bar provides synchronous uncluttered logging, increased stability, and — unintuitively —- speed (due to Python’s GIL). Meanwhile threaded progress bars are more responsive, able to update multiple stdout lines at a time, and can look prettier (unless you try to log stdout to disk).

ProgIter was originally developed independently of tqdm, but the newer versions of this library have been designed to be compatible with tqdm-API. ProgIter is now a (mostly) drop-in alternative to tqdm. The tqdm library may be more appropriate in some cases. The main advantage of ProgIter is that it does not use any python threading, and therefore can be safer with code that makes heavy use of multiprocessing. The reason for this is that threading before forking may cause locks to be duplicated across processes, which may lead to deadlocks (although with tqdm this is very unlikely).

ProgIter is simpler than tqdm, which may be desirable for some applications. However, this also means ProgIter is not as extensible as tqdm. If you want a pretty bar or need something fancy, use tqdm (or rich); if you want useful information about your iteration by default, use progiter.

New in Version 2.0, progiter.ProgressManager, which enables near-seemless toggling between an unthreaded ProgIter backend and a threaded RichProgIter backend. Basic usage is:

from progiter.manager import ProgressManager
pman = ProgressManager()
with pman:
    for item in pman.progiter(range(1000)):
        ...  # do some work

The following gif illustrates this and more complex usage:

https://i.imgur.com/GE1h3kr.gif

Package level documentation can be found at: https://progiter.readthedocs.io/en/latest/

Example

The basic usage of ProgIter is simple and intuitive: wrap a python iterable. The following example wraps a range iterable and reports progress to stdout as the iterable is consumed. The ProgIter object accepts various keyword arguments to modify the details of how progress is measured and reported. See API documentation of the ProgIter class here: https://progiter.readthedocs.io/en/latest/progiter.progiter.html#progiter.progiter.ProgIter

>>> from progiter import ProgIter
>>> def is_prime(n):
...     return n >= 2 and not any(n % i == 0 for i in range(2, n))
>>> for n in ProgIter(range(1000), verbose=2):
>>>     # do some work
>>>     is_prime(n)
0.00%    0/1000... rate=0 Hz, eta=?, total=0:00:00
0.60%    6/1000... rate=76995.12 Hz, eta=0:00:00, total=0:00:00
100.00% 1000/1000... rate=266488.22 Hz, eta=0:00:00, total=0:00:00

For more complex applications is may sometimes be desireable to manually use the ProgIter API. This is done as follows:

>>> from progiter import ProgIter
>>> n = 3
>>> prog = ProgIter(desc='manual', total=n, verbose=3, time_thresh=0)
>>> prog.begin() # Manually begin progress iteration
>>> for _ in range(n):
...     prog.step(inc=1)  # specify the number of steps to increment
>>> prog.end()  # Manually end progress iteration
manual 0.00% 0/3... rate=0 Hz, eta=?, total=0:00:00
manual 33.33% 1/3... rate=5422.82 Hz, eta=0:00:00, total=0:00:00
manual 66.67% 2/3... rate=8907.61 Hz, eta=0:00:00, total=0:00:00
manual 100.00% 3/3... rate=12248.15 Hz, eta=0:00:00, total=0:00:00

By default ProgIter aims to write a line to stdout once every two seconds to minimize its overhead and reduce clutter. Setting this to zero will force it to print on every iteration.

When working with ProgIter in either iterable or manual mode you can use the prog.ensure_newline method to guarantee that the next call you make to stdout will start on a new line. You can also use the prog.set_extra method to update a dynamic “extra” message that is shown in the formatted output. The following example demonstrates this.

>>> from progiter import ProgIter
>>> def is_prime(n):
...     return n >= 2 and not any(n % i == 0 for i in range(2, n))
>>> _iter = range(1000)
>>> prog = ProgIter(_iter, desc='check primes', verbose=2, time_thresh=1e-3)
>>> for n in prog:
>>>     if n == 97:
>>>         print('!!! Special print at n=97 !!!')
>>>     if is_prime(n):
>>>         prog.set_extra('Biggest prime so far: {}'.format(n))
>>>         prog.ensure_newline()
check primes 0.00%    0/1000... rate=0 Hz, eta=?, total=0:00:00
check primes 0.60%    6/1000...Biggest prime so far: 5 rate=79329.39 Hz, eta=0:00:00, total=0:00:00
!!! Special print at n=97 !!!
check primes 75.60%  756/1000...Biggest prime so far: 751 rate=272693.23 Hz, eta=0:00:00, total=0:00:00
check primes 99.30%  993/1000...Biggest prime so far: 991 rate=245852.75 Hz, eta=0:00:00, total=0:00:00
check primes 100.00% 1000/1000...Biggest prime so far: 997 rate=244317.84 Hz, eta=0:00:00, total=0:00:00

Installation

ProgIter can be easily installed via pip.

pip install progiter

Alternatively, the ubelt library ships with its own version of ProgIter. Note that the ubelt version of progiter is distinct (i.e. ubelt actually contains a copy of this library), but the two libraries are generally kept in sync.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

progiter-2.0.0.tar.gz (28.1 kB view details)

Uploaded Source

Built Distribution

progiter-2.0.0-py3-none-any.whl (24.9 kB view details)

Uploaded Python 3

File details

Details for the file progiter-2.0.0.tar.gz.

File metadata

  • Download URL: progiter-2.0.0.tar.gz
  • Upload date:
  • Size: 28.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.6

File hashes

Hashes for progiter-2.0.0.tar.gz
Algorithm Hash digest
SHA256 155b17bf3f090c3791219a2fce37769c96852b38f62ea9baf9162c731e8870e4
MD5 dd4cf8c7f0d6482245fa261db1b54a95
BLAKE2b-256 0242c7ea4b181feadb567ced59636ba72bfd307aa2a00d83c22e38b7d823fc3e

See more details on using hashes here.

File details

Details for the file progiter-2.0.0-py3-none-any.whl.

File metadata

  • Download URL: progiter-2.0.0-py3-none-any.whl
  • Upload date:
  • Size: 24.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.6

File hashes

Hashes for progiter-2.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 66ab3aa88f804e969477ad9de52926a08c2d5b673d324132cfe1530888e26f10
MD5 09dfd71b0472f386beb9dc9452546895
BLAKE2b-256 c4d5981d5651ecf71f47aeeecdb99b175daa1bfb2e157253e8c016a6515bf63e

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page