numexpr

Fast numerical expression evaluator for NumPy

These details have been verified by PyPI

Maintainers

andreabedini falted matthew.brett robbmcleod

These details have not been verified by PyPI

Project links

Homepage

Project description

Author:: David M. Cooke, Francesc Alted, and others.
Maintainer:: Robert A. McLeod
Contact:: robbmcleod@gmail.com
URL:: https://github.com/pydata/numexpr
Documentation:: http://numexpr.readthedocs.io/en/latest/
Travis CI:
GitHub Actions:
PyPi:
DOI:
readthedocs:

What is NumExpr?

NumExpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like '3*a+4*b') are accelerated and use less memory than doing the same calculation in Python.

In addition, its multi-threaded capabilities can make use of all your cores – which generally results in substantial performance scaling compared to NumPy.

Last but not least, numexpr can make use of Intel’s VML (Vector Math Library, normally integrated in its Math Kernel Library, or MKL). This allows further acceleration of transcendent expressions.

How NumExpr achieves high performance

The main reason why NumExpr achieves better performance than NumPy is that it avoids allocating memory for intermediate results. This results in better cache utilization and reduces memory access in general. Due to this, NumExpr works best with large arrays.

NumExpr parses expressions into its own op-codes that are then used by an integrated computing virtual machine. The array operands are split into small chunks that easily fit in the cache of the CPU and passed to the virtual machine. The virtual machine then applies the operations on each chunk. It’s worth noting that all temporaries and constants in the expression are also chunked. Chunks are distributed among the available cores of the CPU, resulting in highly parallelized code execution.

The result is that NumExpr can get the most of your machine computing capabilities for array-wise computations. Common speed-ups with regard to NumPy are usually between 0.95x (for very simple expressions like 'a + 1') and 4x (for relatively complex ones like 'a*b-4.1*a > 2.5*b'), although much higher speed-ups can be achieved for some functions and complex math operations (up to 15x in some cases).

NumExpr performs best on matrices that are too large to fit in L1 CPU cache. In order to get a better idea on the different speed-ups that can be achieved on your platform, run the provided benchmarks.

Installation

From wheels

NumExpr is available for install via pip for a wide range of platforms and Python versions (which may be browsed at: https://pypi-hypernode.com/project/numexpr/#files). Installation can be performed as:

pip install numexpr

If you are using the Anaconda or Miniconda distribution of Python you may prefer to use the conda package manager in this case:

conda install numexpr

From Source

On most Nix systems your compilers will already be present. However if you are using a virtual environment with a substantially newer version of Python than your system Python you may be prompted to install a new version of `gcc or clang.

For Windows, you will need to install the Microsoft Visual C++ Build Tools (which are free) first.The version depends on which version of Python you have installed:

https://wiki.python.org/moin/WindowsCompilers

For Python 3.6+ simply installating the latest version of MSVC build tools should be sufficient. Note that wheels found via pip do not include MKL support. Wheels available via conda will have MKL, if the MKL backend is used for NumPy.

See requirements.txt for the required version of NumPy.

NumExpr is built in the standard Python way:

python setup.py build install

You can test numexpr with:

python -c "import numexpr; numexpr.test()"

Do not test NumExpr in the source directory or you will generate import errors.

Enable Intel® MKL support

NumExpr includes support for Intel’s MKL library. This may provide better performance on Intel architectures, mainly when evaluating transcendental functions (trigonometrical, exponential…).

If you have Intel’s MKL, copy the site.cfg.example that comes with the distribution to site.cfg and edit the latter file to provide correct paths to the MKL libraries in your system. After doing this, you can proceed with the usual building instructions listed above.

Pay attention to the messages during the building process in order to know whether MKL has been detected or not. Finally, you can check the speed-ups on your machine by running the bench/vml_timing.py script (you can play with different parameters to the set_vml_accuracy_mode() and set_vml_num_threads() functions in the script so as to see how it would affect performance).

Usage

>>> import numpy as np
>>> import numexpr as ne

>>> a = np.arange(1e6)   # Choose large arrays for better speedups
>>> b = np.arange(1e6)

>>> ne.evaluate("a + 1")   # a simple expression
array([  1.00000000e+00,   2.00000000e+00,   3.00000000e+00, ...,
         9.99998000e+05,   9.99999000e+05,   1.00000000e+06])

>>> ne.evaluate('a*b-4.1*a > 2.5*b')   # a more complex one
array([False, False, False, ...,  True,  True,  True], dtype=bool)

>>> ne.evaluate("sin(a) + arcsinh(a/b)")   # you can also use functions
array([        NaN,  1.72284457,  1.79067101, ...,  1.09567006,
        0.17523598, -0.09597844])

>>> s = np.array([b'abba', b'abbb', b'abbcdef'])
>>> ne.evaluate("b'abba' == s")   # string arrays are supported too
array([ True, False, False], dtype=bool)

Documentation

Please see the official documentation at numexpr.readthedocs.io. Included is a user guide, benchmark results, and the reference API.

Authors

Please see AUTHORS.txt.

License

NumExpr is distributed under the MIT license.

Project details

These details have been verified by PyPI

Maintainers

andreabedini falted matthew.brett robbmcleod

These details have not been verified by PyPI

Project links

Homepage

Release history Release notifications | RSS feed

2.10.1

Jun 20, 2024

2.10.0

Apr 2, 2024

2.9.0

Jan 26, 2024

2.8.8

Dec 11, 2023

2.8.7

Sep 26, 2023

2.8.6

Sep 12, 2023

2.8.5

Aug 6, 2023

2.8.4

Oct 26, 2022

2.8.3

Jun 25, 2022

2.8.1

Dec 10, 2021

This version

2.8.0

Dec 2, 2021

2.7.3

Mar 3, 2021

2.7.2

Dec 29, 2020

2.7.1

Jan 5, 2020

2.7.0

Aug 14, 2019

2.6.9

Dec 21, 2018

2.6.8

Aug 19, 2018

2.6.7

Aug 12, 2018

2.6.6

Jul 16, 2018

2.6.6.dev0 pre-release yanked

May 1, 2018

Reason this release was yanked:

Not intended to be released

2.6.5

May 1, 2018

2.6.4

Sep 15, 2017

2.6.3

Sep 14, 2017

2.6.2

Jan 29, 2017

2.6.1

Jul 17, 2016

2.6.0

Jun 1, 2016

2.5.2

Apr 7, 2016

2.5.1

Mar 31, 2016

2.5

Feb 6, 2016

2.4.6

Nov 2, 2015

2.4.5

Nov 2, 2015

2.4.4

Sep 14, 2015

2.4.3

Apr 27, 2015

2.4.2

Apr 22, 2015

2.4.1

Apr 14, 2015

2.4

Apr 13, 2014

2.4-rc2 pre-release

Apr 7, 2014

2.4-rc1 pre-release

Apr 6, 2014

2.3.1

Feb 18, 2014

2.3

Jan 26, 2014

2.3-rc1 pre-release

Jan 24, 2014

2.2.2

Sep 27, 2013

2.2.1

Sep 8, 2013

2.2

Aug 31, 2013

2.1

Apr 27, 2013

2.0.1

Jan 8, 2012

2.0

Nov 27, 2011

1.4.2

Jan 25, 2011

1.4.1

Oct 20, 2010

1.4

Aug 1, 2010

1.3.1

Jun 23, 2009

1.3

Jun 2, 2009

1.2

Feb 18, 2009

1.1.1

Jan 23, 2009

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

numexpr-2.8.0.tar.gz (95.8 kB view details)

Uploaded Dec 2, 2021 Source

Built Distributions

numexpr-2.8.0-cp310-cp310-win_amd64.whl (88.5 kB view details)

Uploaded Dec 2, 2021 CPython 3.10 Windows x86-64

numexpr-2.8.0-cp310-cp310-win32.whl (88.2 kB view details)

Uploaded Dec 2, 2021 CPython 3.10 Windows x86

numexpr-2.8.0-cp310-cp310-musllinux_1_1_x86_64.whl (930.7 kB view details)

Uploaded Dec 2, 2021 CPython 3.10 musllinux: musl 1.1+ x86-64

numexpr-2.8.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (380.9 kB view details)

Uploaded Dec 2, 2021 CPython 3.10 manylinux: glibc 2.17+ x86-64

numexpr-2.8.0-cp310-cp310-macosx_10_9_x86_64.whl (98.7 kB view details)

Uploaded Dec 2, 2021 CPython 3.10 macOS 10.9+ x86-64

numexpr-2.8.0-cp39-cp39-win_amd64.whl (88.5 kB view details)

Uploaded Dec 2, 2021 CPython 3.9 Windows x86-64

numexpr-2.8.0-cp39-cp39-win32.whl (88.2 kB view details)

Uploaded Dec 2, 2021 CPython 3.9 Windows x86

numexpr-2.8.0-cp39-cp39-musllinux_1_1_x86_64.whl (929.6 kB view details)

Uploaded Dec 2, 2021 CPython 3.9 musllinux: musl 1.1+ x86-64

numexpr-2.8.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (380.4 kB view details)

Uploaded Dec 2, 2021 CPython 3.9 manylinux: glibc 2.17+ x86-64

numexpr-2.8.0-cp39-cp39-macosx_10_9_x86_64.whl (98.7 kB view details)

Uploaded Dec 2, 2021 CPython 3.9 macOS 10.9+ x86-64

numexpr-2.8.0-cp38-cp38-win_amd64.whl (88.4 kB view details)

Uploaded Dec 2, 2021 CPython 3.8 Windows x86-64

numexpr-2.8.0-cp38-cp38-win32.whl (88.2 kB view details)

Uploaded Dec 2, 2021 CPython 3.8 Windows x86

numexpr-2.8.0-cp38-cp38-musllinux_1_1_x86_64.whl (931.3 kB view details)

Uploaded Dec 2, 2021 CPython 3.8 musllinux: musl 1.1+ x86-64

numexpr-2.8.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (381.4 kB view details)

Uploaded Dec 2, 2021 CPython 3.8 manylinux: glibc 2.17+ x86-64

numexpr-2.8.0-cp38-cp38-macosx_10_9_x86_64.whl (98.7 kB view details)

Uploaded Dec 2, 2021 CPython 3.8 macOS 10.9+ x86-64

numexpr-2.8.0-cp37-cp37m-win_amd64.whl (88.4 kB view details)

Uploaded Dec 2, 2021 CPython 3.7m Windows x86-64

numexpr-2.8.0-cp37-cp37m-win32.whl (88.1 kB view details)

Uploaded Dec 2, 2021 CPython 3.7m Windows x86

numexpr-2.8.0-cp37-cp37m-musllinux_1_1_x86_64.whl (930.5 kB view details)

Uploaded Dec 2, 2021 CPython 3.7m musllinux: musl 1.1+ x86-64

numexpr-2.8.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (379.2 kB view details)

Uploaded Dec 2, 2021 CPython 3.7m manylinux: glibc 2.17+ x86-64

numexpr-2.8.0-cp37-cp37m-macosx_10_9_x86_64.whl (101.7 kB view details)

Uploaded Dec 2, 2021 CPython 3.7m macOS 10.9+ x86-64

numexpr-2.8.0-cp36-cp36m-win_amd64.whl (88.4 kB view details)

Uploaded Dec 2, 2021 CPython 3.6m Windows x86-64

numexpr-2.8.0-cp36-cp36m-win32.whl (88.1 kB view details)

Uploaded Dec 2, 2021 CPython 3.6m Windows x86

numexpr-2.8.0-cp36-cp36m-musllinux_1_1_x86_64.whl (928.3 kB view details)

Uploaded Dec 2, 2021 CPython 3.6m musllinux: musl 1.1+ x86-64

numexpr-2.8.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (378.1 kB view details)

Uploaded Dec 2, 2021 CPython 3.6m manylinux: glibc 2.17+ x86-64

numexpr-2.8.0-cp36-cp36m-macosx_10_9_x86_64.whl (101.7 kB view details)

Uploaded Dec 2, 2021 CPython 3.6m macOS 10.9+ x86-64

File details

Details for the file numexpr-2.8.0.tar.gz.

File metadata

Download URL: numexpr-2.8.0.tar.gz
Upload date: Dec 2, 2021
Size: 95.8 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/3.7.0 importlib_metadata/3.10.0 pkginfo/1.8.2 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.4

File hashes

Hashes for numexpr-2.8.0.tar.gz
Algorithm	Hash digest
SHA256	`9fec076b76c90a5f3929373f548834bb203c6d23a81a895e60d0fe9cca075e99`
MD5	`b92fa723aa3791df2e562a3d7cc051ac`
BLAKE2b-256	`6f184f07b0f5effb3b6fe4611738c15ce208f36dfe140c1f3ba1f28e62294717`

Algorithm	Hash digest
SHA256	`3b429e69572e57ce7476de3bac984f237d980b28bfce72cb610ade0e7ae6442d`
MD5	`79e09ab3243fc6af70c75d5f36d0ebe5`
BLAKE2b-256	`df5d114ef60d97d4cabb6bf10819a328633868cbdffb232c4e1f58729d2b9be8`

Algorithm	Hash digest
SHA256	`217e04783abaedfc8173f3f0aef555db5ce81c210e23f3ea695cc523b41da638`
MD5	`bfd20b9da8ccdf5e6ea02e7c683bc71d`
BLAKE2b-256	`edfd172ab7db3a3997e2d6c67f2c4287a2d9442bb59a900cf88bfaecf2e8f58b`

Algorithm	Hash digest
SHA256	`6e0cc5d65b02eff4aa3d37da005597871915b977800892c149b82495c141b630`
MD5	`e107dcf2707618cfbf25a693647150ea`
BLAKE2b-256	`37beffc822e4e83dde6813cf36080fa106342e3be112f5dde43cc56ece4bbe81`

Algorithm	Hash digest
SHA256	`7c604b695f07e49e6ac58a2eb9dca0dae8b9cb115e3cd6148b28c7be1c49249d`
MD5	`72b76c076afa5d1aa48433cf50d316c2`
BLAKE2b-256	`28173e7a34cf7acde4445545a69356ee1e769cfb2c2b6c74c43ae1dbdeebfa56`

Algorithm	Hash digest
SHA256	`9c99f207cef209f5dd42c7f389753e9ace24b9e306a278264581b5390dd9e7d6`
MD5	`b360c5154514ebe51e0e5e6095a72421`
BLAKE2b-256	`357af476a1820f82bca9304ad3afd35a21387ac5f37b1df132f1945b34a7bc51`

Algorithm	Hash digest
SHA256	`cebc549dbe99eeee4be9a6ab12f19366c14d88686041db3b97093aa60b5834be`
MD5	`cf6e0120994a6bff2b4f1a1e235570f3`
BLAKE2b-256	`460c35b998626dad183402d5a9dd0e50d1d916409a74ae61008fe2f1fad01334`

Algorithm	Hash digest
SHA256	`6e0be6495b92447459f1f0c6b270624a489a1e434a9c8f68950e1450cb825f38`
MD5	`f5e811c5bf2f84c89c460144871884a4`
BLAKE2b-256	`1675e7854d8076ec661d63f93740e166346a4a1d13f0ff7b0511ba2136e07540`

Algorithm	Hash digest
SHA256	`cb2bee19e39c73110b9f12a83ffd1438c13e487ef2237985c0434c67243a4aa5`
MD5	`8d71bea5049f696de0f6c0bbd50b6903`
BLAKE2b-256	`3f34fb45549a06fcffcb78e9dd925f1aa0d1913d9a1c6783149f9105ceb90a7a`

Algorithm	Hash digest
SHA256	`078ae8c6b7028a893f72bd8a88cdda1f12e08dfa1eb67783a41da360524ca5dc`
MD5	`42fca5574567c1baaca74a5ceb0d2f4d`
BLAKE2b-256	`a1d9dc0f6d6b2e4087c2b1e3af6af38297718b21e9a45765e13512e6b01a176b`

Algorithm	Hash digest
SHA256	`2d8cc7181c6bbe315781aed5895a4a5e90d161841ded1deb792be1fd2b1a539f`
MD5	`92793d58e9ac512a52af5487d54508b3`
BLAKE2b-256	`202665431b9df9f18882903ff5fbc8163c1f3a915a42a76c042688f1f2167d46`

Algorithm	Hash digest
SHA256	`d9caca66cfc5bfcba22e94414408bcd89c036d66c63fca313d5bfd6712d2b9de`
MD5	`4d08ea3776f06c4c544841de175e6221`
BLAKE2b-256	`fd1525432603c9fee8b4746c879d773ae126bf860cfc7836028ce9b6b5f7c264`

Algorithm	Hash digest
SHA256	`7143a7e0a5db48d4302e12280f04a886c367f8cea19dccd009be6bbefa4bbc0c`
MD5	`68337bd8948b89c92a83e4355f34126e`
BLAKE2b-256	`9ea99aac016ca261e335a4d63fa34366f0322f30b93220d6807cf0738f793018`

Algorithm	Hash digest
SHA256	`edbb4cbd4c0a1976bd3dff856f88e6477dfd2d843e0927f85cdc964d5c8544f5`
MD5	`344b6b829f9e42c5fcfc5d54af1c1e55`
BLAKE2b-256	`090c6107cb8bc9e3ebc9f6defac770bdc68cd5cfcdadde6c111e381907b1a5fb`

Algorithm	Hash digest
SHA256	`0bdec52a27cc7945cf58650f585bdc3e9089f097190c31cd061dd80b864484bb`
MD5	`39f7945a204fe09aa6c1bc74e0e7f3d8`
BLAKE2b-256	`3e3e65b37d41c037c3d1c188cfb915839edad6694ab163dffa2e4b0839bab466`

Algorithm	Hash digest
SHA256	`09b69583b694466c2f02311a69c35e35e59fed2842decd8926aaf6466478fad6`
MD5	`ba68a0bd744687c28da031c24cf027a9`
BLAKE2b-256	`086f1d8053c9a90805c492f070d3b1607cfcc2ba5b5b5e754abb63b4fe841100`

Algorithm	Hash digest
SHA256	`f7ae9a282bce5b5a184ddc1572bc23fa1e656fb45511a574bccb2b9aad907b1c`
MD5	`5c98e28a332dca918db342629988d059`
BLAKE2b-256	`80827bea402db30db92c675799ec22c2b87524f332e2bbe49e8de352bfa57586`

Algorithm	Hash digest
SHA256	`2c8f548690a302db859f5b79c057d8fe552f95c0f40f36804f9bda02ec660917`
MD5	`00da676353e47b99abc8e7d3a176e350`
BLAKE2b-256	`8892c4a1dcc7f609ae67530e3cdacac3a353818b8ca1d33c3153543dadfc257b`

Algorithm	Hash digest
SHA256	`1150ffee400f23b413c07ed739e2d7e63df2049739535916972769c86f3feb09`
MD5	`7ddc5dddc61eff210a5d5f4ddaaef358`
BLAKE2b-256	`31054bd3e34ebe19c714e9f3cfcfd147be749e7a5647716a597d48da23c1a6a5`

Algorithm	Hash digest
SHA256	`a96293ab134d09aeb3b8b09cf2745ed1debbf3a3aa7071bcca93c9d0c9dcf3f9`
MD5	`af4a95bacd1200e3f4b66c0c7c56cc9e`
BLAKE2b-256	`5d12bee29d112f4e82c6ca74a522f23e00aff6cae777e7444e5323ad2e81c1e6`

Algorithm	Hash digest
SHA256	`ab63acfbe55c05f33dc5d33fde1449434b458a40a1ee9324897c27dc0629794d`
MD5	`939ebfd10b498174a50f2e517ab3f6fd`
BLAKE2b-256	`aeba6111a7d70087906b9650865b515c112b7bd11c1793f7b4e799b3484ff13e`

numexpr 2.8.0

Navigation

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Project description

What is NumExpr?

How NumExpr achieves high performance

Installation

From wheels

From Source

Enable Intel® MKL support

Usage

Documentation

Authors

License

Project details

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Release history Release notifications | RSS feed

Download files

Source Distribution

Built Distributions

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

Algorithm	Hash digest
SHA256	`4979cdea3814dd4519cf413c13ed705114c41f4d5a70d490ef0f4904255e93fd`
MD5	`5fd2914004eac0bb31a7d6d7a9bb1930`
BLAKE2b-256	`799616782dd320285023e1a1abc8b388f23d29fa337c9c66e5053a2cdbe885fd`

Algorithm	Hash digest
SHA256	`4e74eb1ccdd96b7726b34b018c0f210486dacc3de86bb86b808b1abd25b535e6`
MD5	`573ea809ad5ede27387aad18b3fad83f`
BLAKE2b-256	`46837159b13ac59c5fd1d4b9240bf3348b1a393ad347f77f189b672f578fee35`

Algorithm	Hash digest
SHA256	`142dabece2a223cf744f0afe633016ae56d4992a0798258e62da5b503e87ebed`
MD5	`395c8829703c8786e7054d881eb904ac`
BLAKE2b-256	`12f1f7b16cef9cfd78ccfdafd9e3667dabe0840f74cb767dc94795a63b4797a7`

Algorithm	Hash digest
SHA256	`b70369863c5f1fcee22ef8979317067e2baa84659194c733632f58e89bab729f`
MD5	`8d8b81d4e94566f6dacd1544ad8f620a`
BLAKE2b-256	`f24816bfa2ec6bcf957f54fd0af21ed0ecf23edc34995d9d4d95f4b6bcde4179`

Algorithm	Hash digest
SHA256	`3be075577725c23bf1e50501ecc095421ac4e3e5f75bd4842dd1e928cd4f4bdd`
MD5	`7e2ac9252d87488e37337f5f25898d40`
BLAKE2b-256	`c2b3a39444198842915df2dbc77562ccd691c53341865a220fadcf78d8248861`