listful

Description

Efficient filtering of lists of objects

Installation

pip install listful

Usage

Initialize with the fields you want to filter by:

>>> from listful import Listful
>>> data = Listful(
...    [{'x': 1, 'y': 10}, {'x': 2, 'y': 20}, {'x': 2, 'y': 30}], 
...    fields=['x', 'y']
... )

(If you don't specify the fields, all the fields whose corresponding values are hashable will be chosen)

Filtering:

• By one field:

>>> data.filter(x=1).one_or_none()
{'x': 1, 'y': 10}
>>> data.filter(y=20).one_or_none()
{'x': 2, 'y': 20}

• By one field, with more than one result:

>>> data.filter(x=2).to_list()
[{'x': 2, 'y': 20}, {'x': 2, 'y': 30}]

• By two fields:

>>> data.filter(x=2, y=30).one_or_none()
{'x': 2, 'y': 30}

• Raise exception if more than one found

>>> data.filter(x=2).one_or_raise()
Traceback (most recent call last):
<...>
listful.exceptions.MoreThanOneResultException: Found more than one result for filter {'x': 2}: [{'x': 2, 'y': 20}, {'x': 2, 'y': 30}]

• Get all values for a specific field

>>> data.get_all_for_field('x')
[1, 2, 2]

Updating indexes:

Listful has the same api as list, so you can get/set/delete items the same way and the indices will be updated automatically

>>> data[0] = {'x': 17, 'y': 17}
>>> data.filter(x=17).one_or_none()
{'x': 17, 'y': 17}
>>> data[0]
{'x': 17, 'y': 17}
>>> del data[0]
>>> data.filter(x=17).one_or_none()

If you want to modify an element and update the indices you can do so explicitly:

>>> data[0]['x'] = 1
>>> data.rebuild_indexes_for_item(data[0])
>>> data.filter(x=1).one_or_none()
{'x': 1, 'y': 20}

Objects:

Listful supports also lists of objects:

>>> class Item:
...     def __init__(self, x, y):
...         self.x = x
...         self.y = y
...
...     def __repr__(self):
...         return f"Item(x={self.x}, y={self.y})"

>>> items = Listful(
...    [Item(x=1, y=10), Item(x=2, y=20), Item(x=2, y=30)], 
...    fields=['x', 'y']
... )
>>> items.filter(x=1).one_or_none()
Item(x=1, y=10)

Here too, if you don't specify the fields, all fields with hashable values will be chosen:

>>> items = Listful(
...    [Item(x=1, y=10), Item(x=2, y=20), Item(x=2, y=30)], 
... )
>>> items.fields
['x', 'y']

Performance

See scripts/timing.py.

A comparison of filtering with listful vs filtering with pandas (with/without index)

70x faster than pandas with indexing, 360x faster than pandas without indexing.

For developers

Create venv and install deps

make init

Install git precommit hook

make precommit_install

Run linters, autoformat, tests etc.

make pretty lint test

Bump new version

make bump_major
make bump_minor
make bump_patch

License

MIT

Change Log

Unreleased

• ...

0.2.1 - 2020-04-08

• ...

0.2.0 - 2020-04-08

• Add support for default fields

0.1.3 - 2020-02-14

• ...

0.1.1 - 2020-02-12

• ...

0.1.0 - 2020-02-12

• initial