Manipulate JSON-like data with NumPy-like idioms.

These details have not been verified by PyPI

Project links

Project description

Awkward Array is a library for nested, variable-sized data, including arbitrary-length lists, records, mixed types, and missing data, using NumPy-like idioms.

Arrays are dynamically typed, but operations on them are compiled and fast. Their behavior coincides with NumPy when array dimensions are regular and generalizes when they're not.

Motivating example

Given an array of objects with x, y fields and variable-length nested lists like

array = ak.Array([
    [{"x": 1.1, "y": [1]}, {"x": 2.2, "y": [1, 2]}, {"x": 3.3, "y": [1, 2, 3]}],
    [],
    [{"x": 4.4, "y": {1, 2, 3, 4]}, {"x": 5.5, "y": [1, 2, 3, 4, 5]}]
])

the following slices out the y values, drops the first element from each inner list, and runs NumPy's np.square function on everything that is left:

output = np.square(array["y", ..., 1:])

The result is

[
    [[], [4], [4, 9]],
    [],
    [[4, 9, 16], [4, 9, 16, 25]]
]

The equivalent using only Python is

output = []
for sublist in array:
    tmp1 = []
    for record in sublist:
        tmp2 = []
        for number in record["y"][1:]:
            tmp2.append(np.square(number))
        tmp1.append(tmp2)
    output.append(tmp1)

Not only is the expression using Awkward Arrays more concise, using idioms familiar from NumPy, but it's much faster and uses less memory.

For a similar problem 10 million times larger than the one above (on a single-threaded 2.2 GHz processor),

the Awkward Array one-liner takes 4.6 seconds to run and uses 2.1 GB of memory,
the equivalent using Python lists and dicts takes 138 seconds to run and uses 22 GB of memory.

Speed and memory factors in the double digits are common because we're replacing Python's dynamically typed, pointer-chasing virtual machine with type-specialized, precompiled routines on contiguous data. (In other words, for the same reasons as NumPy.) Even higher speedups are possible when Awkward Array is paired with Numba.

Our presentation at SciPy 2020 provides a good introduction, showing how to use these arrays in a real analysis.

Installation

Awkward Array can be installed from PyPI using pip:

pip install awkward1

You will likely get a precompiled binary (wheel), depending on your operating system and Python version. If not, pip attempts to compile from source (which requires a C++ compiler, make, and CMake).

Awkward Array is also available using conda, which always installs a binary:

conda install -c conda-forge awkward1

If you have already added conda-forge as a channel, the -c conda-forge is unnecessary. Adding the channel is recommended because it ensures that all of your packages use compatible versions:

conda config --add channels conda-forge
conda update --all

Getting help

How-to tutorials

Python API reference

C++ API reference

Report bugs, request features, and ask for additional documentation on GitHub Issues.
If you have a "How do I...?" question, ask about it on StackOverflow with the [awkward-array] tag. Be sure to include tags for any other libraries that you use, such as Pandas or PyTorch.
To ask questions in real time, try the Gitter Scikit-HEP/awkward-array chat room.

Project details

These details have not been verified by PyPI

Project links

Release history Release notifications | RSS feed

1.0.0

Dec 5, 2020

1.0.0rc1 pre-release

Dec 1, 2020

This version

0.4.5

Nov 24, 2020

0.4.4

Nov 17, 2020

0.4.3

Nov 5, 2020

0.4.2

Nov 4, 2020

0.4.1

Nov 3, 2020

0.4.0

Oct 29, 2020

0.3.1

Sep 21, 2020

0.3.0

Sep 21, 2020

0.2.38

Sep 18, 2020

0.2.37

Sep 12, 2020

0.2.36

Sep 8, 2020

0.2.35

Aug 26, 2020

0.2.34

Aug 25, 2020

0.2.33

Aug 22, 2020

0.2.32

Aug 19, 2020

0.2.31

Aug 15, 2020

0.2.30

Aug 11, 2020

0.2.29

Aug 10, 2020

0.2.28

Aug 7, 2020

0.2.27

Jul 23, 2020

0.2.26

Jul 23, 2020

0.2.25

Jul 11, 2020

0.2.24

Jun 12, 2020

0.2.23

Jun 6, 2020

0.2.22

May 29, 2020

0.2.21

May 28, 2020

0.2.20

May 14, 2020

0.2.19

May 8, 2020

0.2.10

Apr 7, 2020

0.2.0

Mar 6, 2020

0.1.128

Feb 29, 2020

0.1.92

Jan 28, 2020

0.1.38

Dec 28, 2019

0.1.25

Nov 27, 2019

0.1.17

Oct 22, 2019

0.1.7

Sep 26, 2019

0.1.0

Aug 27, 2019

0.0.14

Aug 17, 2019

0.0.10

Aug 16, 2019

0.0.6

Aug 16, 2019

0.0.1

Aug 16, 2019

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

awkward1-0.4.5.tar.gz (794.4 kB view details)

Uploaded Nov 24, 2020 Source

Built Distributions

awkward1-0.4.5-cp39-cp39-win_amd64.whl (9.8 MB view details)

Uploaded Nov 24, 2020 CPython 3.9 Windows x86-64

awkward1-0.4.5-cp39-cp39-win32.whl (7.3 MB view details)

Uploaded Nov 24, 2020 CPython 3.9 Windows x86

awkward1-0.4.5-cp39-cp39-manylinux2010_x86_64.whl (6.2 MB view details)

Uploaded Nov 24, 2020 CPython 3.9 manylinux: glibc 2.12+ x86-64

awkward1-0.4.5-cp39-cp39-manylinux1_x86_64.whl (5.8 MB view details)

Uploaded Nov 24, 2020 CPython 3.9

awkward1-0.4.5-cp39-cp39-manylinux1_i686.whl (5.9 MB view details)

Uploaded Nov 24, 2020 CPython 3.9

awkward1-0.4.5-cp39-cp39-macosx_10_9_x86_64.whl (6.0 MB view details)

Uploaded Nov 24, 2020 CPython 3.9 macOS 10.9+ x86-64

awkward1-0.4.5-cp38-cp38-win_amd64.whl (9.8 MB view details)

Uploaded Nov 24, 2020 CPython 3.8 Windows x86-64

awkward1-0.4.5-cp38-cp38-win32.whl (7.3 MB view details)

Uploaded Nov 24, 2020 CPython 3.8 Windows x86

awkward1-0.4.5-cp38-cp38-manylinux2010_x86_64.whl (6.2 MB view details)

Uploaded Nov 24, 2020 CPython 3.8 manylinux: glibc 2.12+ x86-64

awkward1-0.4.5-cp38-cp38-manylinux1_x86_64.whl (5.8 MB view details)

Uploaded Nov 24, 2020 CPython 3.8

awkward1-0.4.5-cp38-cp38-manylinux1_i686.whl (5.9 MB view details)

Uploaded Nov 24, 2020 CPython 3.8

awkward1-0.4.5-cp38-cp38-macosx_10_9_x86_64.whl (6.0 MB view details)

Uploaded Nov 24, 2020 CPython 3.8 macOS 10.9+ x86-64

awkward1-0.4.5-cp37-cp37m-win_amd64.whl (9.8 MB view details)

Uploaded Nov 24, 2020 CPython 3.7m Windows x86-64

awkward1-0.4.5-cp37-cp37m-win32.whl (7.3 MB view details)

Uploaded Nov 24, 2020 CPython 3.7m Windows x86

awkward1-0.4.5-cp37-cp37m-manylinux2010_x86_64.whl (6.2 MB view details)

Uploaded Nov 24, 2020 CPython 3.7m manylinux: glibc 2.12+ x86-64

awkward1-0.4.5-cp37-cp37m-manylinux1_x86_64.whl (5.9 MB view details)

Uploaded Nov 24, 2020 CPython 3.7m

awkward1-0.4.5-cp37-cp37m-manylinux1_i686.whl (5.9 MB view details)

Uploaded Nov 24, 2020 CPython 3.7m

awkward1-0.4.5-cp37-cp37m-macosx_10_9_x86_64.whl (5.9 MB view details)

Uploaded Nov 24, 2020 CPython 3.7m macOS 10.9+ x86-64

awkward1-0.4.5-cp36-cp36m-win_amd64.whl (9.8 MB view details)

Uploaded Nov 24, 2020 CPython 3.6m Windows x86-64

awkward1-0.4.5-cp36-cp36m-win32.whl (7.3 MB view details)

Uploaded Nov 24, 2020 CPython 3.6m Windows x86

awkward1-0.4.5-cp36-cp36m-manylinux2010_x86_64.whl (6.2 MB view details)

Uploaded Nov 24, 2020 CPython 3.6m manylinux: glibc 2.12+ x86-64

awkward1-0.4.5-cp36-cp36m-manylinux1_x86_64.whl (5.9 MB view details)

Uploaded Nov 24, 2020 CPython 3.6m

awkward1-0.4.5-cp36-cp36m-manylinux1_i686.whl (5.9 MB view details)

Uploaded Nov 24, 2020 CPython 3.6m

awkward1-0.4.5-cp35-cp35m-win_amd64.whl (9.8 MB view details)

Uploaded Nov 24, 2020 CPython 3.5m Windows x86-64

awkward1-0.4.5-cp35-cp35m-win32.whl (7.3 MB view details)

Uploaded Nov 24, 2020 CPython 3.5m Windows x86

awkward1-0.4.5-cp35-cp35m-manylinux2010_x86_64.whl (6.2 MB view details)

Uploaded Nov 24, 2020 CPython 3.5m manylinux: glibc 2.12+ x86-64

awkward1-0.4.5-cp35-cp35m-manylinux1_x86_64.whl (5.9 MB view details)

Uploaded Nov 24, 2020 CPython 3.5m

awkward1-0.4.5-cp35-cp35m-manylinux1_i686.whl (5.9 MB view details)

Uploaded Nov 24, 2020 CPython 3.5m

awkward1-0.4.5-cp27-cp27mu-manylinux2010_x86_64.whl (6.2 MB view details)

Uploaded Nov 24, 2020 CPython 2.7mu manylinux: glibc 2.12+ x86-64

awkward1-0.4.5-cp27-cp27mu-manylinux1_x86_64.whl (5.9 MB view details)

Uploaded Nov 24, 2020 CPython 2.7mu

awkward1-0.4.5-cp27-cp27mu-manylinux1_i686.whl (5.9 MB view details)

Uploaded Nov 24, 2020 CPython 2.7mu

awkward1-0.4.5-cp27-cp27m-win_amd64.whl (9.8 MB view details)

Uploaded Nov 24, 2020 CPython 2.7m Windows x86-64

awkward1-0.4.5-cp27-cp27m-win32.whl (7.3 MB view details)

Uploaded Nov 24, 2020 CPython 2.7m Windows x86

awkward1-0.4.5-cp27-cp27m-manylinux2010_x86_64.whl (6.2 MB view details)

Uploaded Nov 24, 2020 CPython 2.7m manylinux: glibc 2.12+ x86-64

awkward1-0.4.5-cp27-cp27m-manylinux1_x86_64.whl (5.9 MB view details)

Uploaded Nov 24, 2020 CPython 2.7m

awkward1-0.4.5-cp27-cp27m-manylinux1_i686.whl (5.9 MB view details)

Uploaded Nov 24, 2020 CPython 2.7m

awkward1-0.4.5-cp27-cp27m-macosx_10_9_x86_64.whl (5.9 MB view details)

Uploaded Nov 24, 2020 CPython 2.7m macOS 10.9+ x86-64

File details

Details for the file awkward1-0.4.5.tar.gz.

File metadata

Download URL: awkward1-0.4.5.tar.gz
Upload date: Nov 24, 2020
Size: 794.4 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/3.2.0 pkginfo/1.6.1 requests/2.25.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.53.0 CPython/3.7.9

File hashes

Hashes for awkward1-0.4.5.tar.gz
Algorithm	Hash digest
SHA256	`8c786d1641343e9887c21794f9a3bf51716b00ed3b182491f1acbc9e389b31bb`
MD5	`7fd5d5ad430d1f2143918b647354f7fd`
BLAKE2b-256	`959705fe8f2ab72f4201385c972b06329a77399eee3e735a8adef690b61f0392`

Algorithm	Hash digest
SHA256	`1860d885c822332db60f6b7f94c2f4f60d2f59736eb4fbaa09302dd53467329d`
MD5	`22985b157d134f66b33806d85adfbf90`
BLAKE2b-256	`e072c9f8d59f25983b86f2ede1abe70782969e6c93011cad2de20b1eba7e3a4a`

Algorithm	Hash digest
SHA256	`52b02c3f29fece982f97f2fc8fd3ee6149846bcc5b314886f9736f8e508bb47a`
MD5	`8c117fb45321a4cded31e277ba223870`
BLAKE2b-256	`ed0f108fcc6c8b73e4336140f45d088bbe67b25dbd2f6b4dc7da86c00a6b6b3d`

Algorithm	Hash digest
SHA256	`55da391504a50e923c24f552286d61e285da7276576812e42b7f8655fb17e618`
MD5	`9b93e2c0969e62b7c810825fdc4be39e`
BLAKE2b-256	`be83856d9228a98d20afd0d66e0e2acd2db70581b9c59334ae2b1bf9b5184218`

Algorithm	Hash digest
SHA256	`f837de85fd2d26d8b94d567a19048c03f734ee24b13380b6249ab3e852b8a49b`
MD5	`d60c0e39578f9270a090c36d7697d395`
BLAKE2b-256	`22f1c449475cea0d025bf10a6640839e04804c05ab02f10e6859c1436c9c6d4d`

Algorithm	Hash digest
SHA256	`259b0f6ed461b444ec60ccd965d18c216342276ba17fc316d99910aecc2b3690`
MD5	`516c9e078596c5ea0fcaa212220ffb04`
BLAKE2b-256	`29a5a04769f4bf1c942efaa8f85171c87a4656a0a8f3ea960987de8454d6e10d`

Algorithm	Hash digest
SHA256	`18aca87b95cb5b3d188455fe50d5944a3f86a3f03f8899e0d1bf3be88e91dd5c`
MD5	`c8073691416353541ad5f419aee21645`
BLAKE2b-256	`7d73d1affe9930a9d0d6750f1b6db79600749f126cab1786048d16a44ff17c8c`

Algorithm	Hash digest
SHA256	`75bc3296491236b27ec3b41dde6300927253e7d2b314fb50eb69ed522d0651f9`
MD5	`ab47201ce24bea172c243bac7c58a69e`
BLAKE2b-256	`3ee5d6f3fc5496cf85c4a66a716c830d595953e484f18c9d47c1f9263cc2d9a1`

Algorithm	Hash digest
SHA256	`0abe01d88e9b2d90950a2939bacc6ca352864a53e87d5129865f4576412d767a`
MD5	`add64f28c7ecd88e87917325c800dc05`
BLAKE2b-256	`c28175ec7c3ef035d72a23ceb3012e0704adf4b4d769477b6b309aed55060af5`

Algorithm	Hash digest
SHA256	`37df18ce182cbce9ac0f84f737428a9c2c5024f458f5a1582ac3270fb9521995`
MD5	`c018a23d23620db76fb5f3e50fdaa280`
BLAKE2b-256	`aaec58e8c1a5ce407befd48b576eead84ba1ae1a4a0e5458224a02657298f31d`

Algorithm	Hash digest
SHA256	`b9b38f1c78ebb4f8accb54b309872fa6293672a2c6a936530cfa493a5a91d963`
MD5	`613ba4886d0b3db99750c3da73295bfe`
BLAKE2b-256	`92f2b49ce66e7e828a5ceaa3783dd0738a399fb88985280e23d0972c08dcfb15`

Algorithm	Hash digest
SHA256	`ec561869380f46bf531e5a0eaf35e753386547548b4b57a19b66ccf5fae314c5`
MD5	`89a6d7537bbe3410dfa796e5612d8ca0`
BLAKE2b-256	`989e5cabd120437c6631f86c1ba59f40732fa69dd5e8322e1536bf7896d5d4be`

Algorithm	Hash digest
SHA256	`6fd4685eb7a45141e1ee8efce1ec6b069719eb93b1cffc4a30524198c8bab00a`
MD5	`60ed8eccc15e341f2e731668920ac87c`
BLAKE2b-256	`7bcd33c4d66fe4ad62ed427ff5101e9d4a762bf85c74e36155b8e5e43f05a560`

Algorithm	Hash digest
SHA256	`23be6fc0010c5c460b6c5dd03a04cd6bcab69272e3ad292a87228dd455cb2c2f`
MD5	`5a21fa67a2f11741dd5ceaa5f2d0b464`
BLAKE2b-256	`a0797a99b9e2e878ed1d670e57430f7f147c7f00f8f2f30d74231ab03d24411e`

Algorithm	Hash digest
SHA256	`a4f6e9f1fee5df10b5c4f56d7a74f0bf3edcc4a337fe8759f44b6339fa4067d7`
MD5	`0ef976827d825f4ae5131861957cdd02`
BLAKE2b-256	`ca33b5767faad3b6a2d2253bc46aecf7777ce29f357fdc47cea4a05dbbbd68ea`

Algorithm	Hash digest
SHA256	`1977e4bb5ff68746d4a08e9eae8bacd7cc6ea3328cde7a0f8ced0df101b1f2d6`
MD5	`bb17f702a5dda761de1b3e33f4bfd662`
BLAKE2b-256	`878516f35570fd3b3c9fdc3a1f6f0d3800348c11b802535d3e8c6426c728c347`

Algorithm	Hash digest
SHA256	`16fc57b13a7171524eeef09453e0833ae9fa8add62374b4f097352b9a0bf989e`
MD5	`e7555813684b0109cb0d8c16e19deecd`
BLAKE2b-256	`fbe3c6e4586c59fea76d29e39a5ac20dd1746c7e02f7b41102355879d80a284c`

Algorithm	Hash digest
SHA256	`d99e51359782c9945c59a1ba23e96788b6fa1110eccec5b83ee50bfc496e40a0`
MD5	`9bb70b9a0ad716a901ed06db331bbb91`
BLAKE2b-256	`1e9b5cd26ca5bb097f6616db868f2a136de0e22d4fbb1993512370498e8d6ca0`

Algorithm	Hash digest
SHA256	`f2f35e9353aa34280f13f245ebcd3a67ac49fe9b93af096648540147b4dd51a5`
MD5	`10be11c5f592e7ae4a3ec68b33164067`
BLAKE2b-256	`63855973a4460e7fd2524567fb447005dc817271f4b6cfe243adfcdd098ade0d`

Algorithm	Hash digest
SHA256	`ba5f3ef00b205557311e9b7b49e467a0074f55953330b5fb26a382b2137fe0ef`
MD5	`300c36afb08afd7f4082b5eb463955d2`
BLAKE2b-256	`a0c202ccb7355561bb0ac99802414062d1808dcb33092e99d8a4a954e577bc99`

Algorithm	Hash digest
SHA256	`2652f6eafc1a5438ef7d150cdddd93078c3992161827811207790d52b701581d`
MD5	`d4622d1c2bddaa535df716f25e43a6a8`
BLAKE2b-256	`cc9b55d28eb1e0cbdea6576361b1742cd7d08e802868901c0ae20581d95e1688`

awkward1 0.4.5

Navigation

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Project description

Motivating example

Installation

Getting help

Project details

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Release history Release notifications | RSS feed

Download files

Source Distribution

Built Distributions

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

Algorithm	Hash digest
SHA256	`27dd38e45878b36ea13a7e920ef1b542a89544062ea543ab85f1d34b251e82cc`
MD5	`fdeda40f54b50b441c20f5a2aee3348c`
BLAKE2b-256	`9b9cadd739f2121612324c3f7bb274720d865c507e2fce9854e8869c42db7c64`

Algorithm	Hash digest
SHA256	`9410f7aa92d5fef9074150b4ff295fe23daf430c488f3b46b438ab1d44966952`
MD5	`a907aea609e121d806a09918773ea78c`
BLAKE2b-256	`e5d2eb656563b502158af9c322933b99a9507d35103395948011d23b07a724d1`

Algorithm	Hash digest
SHA256	`8e90caac8ff69d94650aa505331a8eb518ebc983b766c02e989c8a47738e097e`
MD5	`7460a1c60ec581f6c3e74f96b87629ae`
BLAKE2b-256	`6e1c581f3992946908de8f1580b32ee76d381220b40c5f03cce6bea22ede9378`

Algorithm	Hash digest
SHA256	`36284748e661d5812cbb862781959b34ae02c6e2f175ce5de76fd4a30566af50`
MD5	`4ab75ff2ad4fc27196aeaf52db69556d`
BLAKE2b-256	`415d46f0f3edd804ae9d69ff066a8165fd991719c9d3f54c6c34bf3cb6fe1ab1`

Algorithm	Hash digest
SHA256	`f7bee0f7c4415684f8f194928368059c126f9fe0aa1303b8729b9c2601aca48d`
MD5	`d9e7dd2cdce53e8373ea9625158d4c29`
BLAKE2b-256	`3586925debcada7357157e6d288f5251674680e2c56b378ff824bff2acde4a57`

Algorithm	Hash digest
SHA256	`f1d9a11928ca0197b732f043708bbcf3b39ae90627efaa2eb91eba975fc888df`
MD5	`efbf386d4dffa5c91911f3ce159a9875`
BLAKE2b-256	`51ce4c5a4a5c8161527765a0d4843296b412b9105b3475ba068d3f2d79cf283a`

Algorithm	Hash digest
SHA256	`149487b34a55bf5847a078ab48e8a5accfa0fbe67b2aafd58daf9bd68102d834`
MD5	`6bdad3e163fb962834e931b92b1d1e1c`
BLAKE2b-256	`786b414985c629426eb2e7ff391ccbd744c090ecb775b3fb011f172e7335301d`

Algorithm	Hash digest
SHA256	`5f8cad2ffd89cf36655bcddb0bb99a02ef1ead5a69065abd5d9c91e45e859bca`
MD5	`c071842ace1cc681d4d85b08f866348f`
BLAKE2b-256	`3bee19f54af49023d76cd27d64f2b0bb474ec8805d7a231135a714517387fbc7`

Algorithm	Hash digest
SHA256	`9796e3c327b16777e3155362f39f12b21228ffff9694212d5f16b30ad9eddb29`
MD5	`3265853c7d9342de1d9d84a353abea6d`
BLAKE2b-256	`81f168919dc787c6b3b82f7b530f10e016a1c52c03f39824ab846b8bcb1216f9`

Algorithm	Hash digest
SHA256	`844cf792998c1c5b010ede4a325c74e3f2be7fa8c1193397d06fbdb92a675613`
MD5	`5b9b7a08e4493d3c6df318037e6d61fa`
BLAKE2b-256	`65a8e7744cee3af32b5e5fe76ae28ce1200bec4e3a8267359f1893da2bdd6dae`