Skip to main content

SONATA files reader

Project description

banner

license coverage documentation status

libsonata

C++ / Python reader for SONATA circuit files: SONATA guide

Installation

Installing from PyPI

pip install libsonata

Installing as a Python package, directly from GitHub

pip install git+https://github.com/BlueBrain/libsonata

Building the C++ library

git clone git@github.com:BlueBrain/libsonata.git --recursive
cd libsonata
mkdir build && cd build
cmake  -DCMAKE_BUILD_TYPE=Release  -DEXTLIB_FROM_SUBMODULES=ON ..
make -j

Usage (Python)

Nodes

NodeStorage
>>> import libsonata

>>> nodes = libsonata.NodeStorage('path/to/H5/file')

# list populations
>>> nodes.population_names

# open population
>>> population = nodes.open_population(<name>)
NodePopulation
# total number of nodes in the population
>>> population.size

# attribute names
>>> population.attribute_names

# get attribute value for single node, say 42
>>> population.get_attribute('mtype', 42)

# ...or Selection of nodes (see below) => returns NumPy array with corresponding values
>>> selection = libsonata.Selection(values=[1, 5, 9, 42])  # nodes 1, 5, 9, 42
>>> mtypes = population.get_attribute('mtype', selection)
>>> list(zip(selection.flatten(), mtypes))
[(1, u'mtype_of_1'), (5, u'mtype_of_5'), (9, u'mtype_of_9'), (42, u'mtype_of_42')]
Selection

List of element IDs (either node_id, or edge_id) where adjacent IDs are grouped for the sake of efficient HDF5 file access. For instance, {1, 2, 3, 5} sequence becomes {[1, 4), [5, 6)}.

Selection can be instantiated from:
  • a sequence of scalar values (works for NumPy arrays as well)

  • a sequence of pairs (interpreted as ranges above, works for N x 2 NumPy arrays as well)

EdgePopulation connectivity queries (see below) return Selections as well.

>>> selection = libsonata.Selection([1, 2, 3, 5])
>>> selection.ranges
[(1, 4), (5, 6)]
>>> selection = libsonata.Selection([(1, 4), (5, 6)])
>>> selection.flatten()
[1, 2, 3, 5]
>>> selection.flat_size
4
>>> bool(selection)
True

Edges

EdgeStorage

Population handling for EdgeStorage is analogous to NodeStorage:

>>> edges = libsonata.EdgeStorage('path/to/H5/file')

# list populations
>>> edges.population_names

# open population
>>> population = edges.open_population(<name>)
EdgePopulation
# total number of edges in the population
>>> population.size

# attribute names
>>> population.attribute_names

# get attribute value for single edge, say 123
>>> population.get_attribute('delay', 123)

# ...or Selection of edges => returns NumPy array with corresponding values
>>> selection = libsonata.Selection([1, 5, 9])
>>> population.get_attribute('delay', selection) # returns delays for edges 1, 5, 9

…with additional methods for querying connectivity, where the results are selections that can be applied like above

# get source / target node ID for the 42nd edge:
>>> population.source_node(42)
>>> population.target_node(42)

# query connectivity (result is Selection object)
>>> selection_to_1 = population.afferent_edges(1)  # all edges with target node_id 1
>>> population.target_nodes(selection_to_1)  # since selection only contains edges
                                             # targeting node_id 1 the result will be a
                                             # numpy array of all 1's
>>> selection_from_2 = population.efferent_edges(2)  # all edges sourced from node_id 2
>>> selection = population.connecting_edges(2, 1)  # this selection is all edges from
                                                   # node_id 2 to node_id 1

# ...or their vectorized analogues
>>> selection = population.afferent_edges([1, 2, 3])
>>> selection = population.efferent_edges([1, 2, 3])
>>> selection = population.connecting_edges([1, 2, 3], [4, 5, 6])

Reports

SpikeReader
>>> import libsonata

>>> spikes = libsonata.SpikeReader('path/to/H5/file')

# list populations
>>> spikes.get_populations_names()

# open population
>>> population = spikes['<name>']
SpikePopulation
# get all spikes [(node_id, timestep)]
>>> population.get()
[(5, 0.1), (2, 0.2), (3, 0.3), (2, 0.7), (3, 1.3)]

# get all spikes betwen tstart and tstop
>>> population.get(tstart=0.2, tstop=1.0)
[(2, 0.2), (3, 0.3), (2, 0.7)]

# get spikes attribute sorting (by_time, by_id, none)
>>> population.sorting
'by_time'

Pandas can be used to create a dataframe and get a better representation of the data
>>> import pandas

data = population.get()
df = pandas.DataFrame(data=data, columns=['ids', 'times']).set_index('times')
print(df)
       ids
times
0.1      5
0.2      2
0.3      3
0.7      2
1.3      3
SomaReportReader
>>> somas = libsonata.SomaReportReader('path/to/H5/file')

# list populations
>>> somas.get_populations_names()

# open population
>>> population_somas = somas['<name>']
SomaReportPopulation
# get times (tstart, tstop, dt)
>>> population_somas.times
(0.0, 1.0, 0.1)

# get unit attributes
>>> population_somas.time_units
'ms'
>>> population_somas.data_units
'mV'

# node_ids sorted?
>>> population_somas.sorted
True

# get a list of all node ids in the selected population
>>> population_somas.get_node_ids()
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]

# get the DataFrame of the node_id values for the timesteps between tstart and tstop
>>> data_frame = population_somas.get(node_ids=[13, 14], tstart=0.8, tstop=1.0)

# get the data values
>>> data_frame.data
[[13.8, 14.8], [13.9, 14.9]]

# get the list of timesteps
>>> data_frame.times
[0.8, 0.9]

# get the list of node ids
>>> data_frame.ids
[13, 14]

Once again, pandas can be used to create a dataframe using the data, ids and times lists

>>> import pandas

df = pandas.DataFrame(data_frame.data, columns=data_frame.ids, index=data_frame.times)
print(df)
       13    14
0.8  13.8  14.8
0.9  13.9  14.9
ElementReportReader
>>> elements = libsonata.ElementReportReader('path/to/H5/file')

# list populations
>>> elements.get_populations_names()

# open population
>>> population_elements = elements['<name>']
ElementReportPopulation
# get times (tstart, tstop, dt)
>>> population_elements.times
(0.0, 4.0, 0.2)

>>> population_elements.get_node_ids()
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]

# get the DataFrame of the node_id values for the timesteps between tstart and tstop
>>> data_frame = population_elements.get(node_ids=[13, 14], tstart=0.8, tstop=1.0)

# get the data values (list of list of floats with data[time_index][element_index])
>>> data_frame.data
[[46.0, 46.1, 46.2, 46.3, 46.4, 46.5, 46.6, 46.7, 46.8, 46.9], [56.0, 56.1, 56.2, 56.3, 56.4, 56.5, 56.6, 56.7, 56.8, 56.9]]

# get the list of timesteps
>>> data_frame.times
[0.8, 1.0]

# get the list of (node id, element_id)
>>> data_frame.ids
[(13, 30), (13, 30), (13, 31), (13, 31), (13, 32), (14, 32), (14, 33), (14, 33), (14, 34), (14, 34)]

The same way than with spikes and soma reports, pandas can be used to get a better representation of the data

>>> import pandas

df = pandas.DataFrame(data_frame.data, columns=pandas.MultiIndex.from_tuples(data_frame.ids), index=data_frame.times)
print(df)
       13                            14
       30    30    31    31    32    32    33    33    34    34
0.8  46.0  46.1  46.2  46.3  46.4  46.5  46.6  46.7  46.8  46.9
1.0  56.0  56.1  56.2  56.3  56.4  56.5  56.6  56.7  56.8  56.9

For big datasets, using numpy arrays could greatly improve the performance

>>> import numpy

np_data = numpy.asarray(data_frame.data)
np_ids = numpy.asarray(data_frame.ids).T
np_times = numpy.asarray(data_frame.times)

df = pandas.DataFrame(np_data, columns=pandas.MultiIndex.from_arrays(np_ids), index=np_times)

Acknowledgements

The development of this software was supported by funding to the Blue Brain Project, a research center of the École polytechnique fédérale de Lausanne (EPFL), from the Swiss government’s ETH Board of the Swiss Federal Institutes of Technology.

This project/research has received funding from the European Union’s Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 785907 (Human Brain Project SGA2).

License

libsonata is distributed under the terms of the GNU Lesser General Public License version 3, unless noted otherwise, for example, for external dependencies. Refer to COPYING.LESSER and COPYING files for details.

Copyright (c) 2018-2021 Blue Brain Project/EPFL

libsonata is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License version 3 as published by the Free Software Foundation.

libsonata is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with libsonata. If not, see <https://www.gnu.org/licenses/>.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

libsonata-0.1.12.tar.gz (945.5 kB view details)

Uploaded Source

Built Distributions

libsonata-0.1.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

libsonata-0.1.12-cp310-cp310-macosx_10_13_x86_64.whl (1.9 MB view details)

Uploaded CPython 3.10 macOS 10.13+ x86-64

libsonata-0.1.12-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

libsonata-0.1.12-cp39-cp39-macosx_10_13_x86_64.whl (1.9 MB view details)

Uploaded CPython 3.9 macOS 10.13+ x86-64

libsonata-0.1.12-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

libsonata-0.1.12-cp38-cp38-macosx_10_13_x86_64.whl (1.9 MB view details)

Uploaded CPython 3.8 macOS 10.13+ x86-64

libsonata-0.1.12-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

libsonata-0.1.12-cp37-cp37m-macosx_10_13_x86_64.whl (1.9 MB view details)

Uploaded CPython 3.7m macOS 10.13+ x86-64

File details

Details for the file libsonata-0.1.12.tar.gz.

File metadata

  • Download URL: libsonata-0.1.12.tar.gz
  • Upload date:
  • Size: 945.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.63.0 importlib-metadata/4.11.2 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.10

File hashes

Hashes for libsonata-0.1.12.tar.gz
Algorithm Hash digest
SHA256 f0fa0f3b129d28e41b337ce2c39c3604990752de8e485327ec9df3bf0360e9c1
MD5 edf72af3dd9f2af12815a08afdbb85ee
BLAKE2b-256 632d814ee9765ebdf0ad07257aa429e1a39f962b974783375395e150bfda4801

See more details on using hashes here.

File details

Details for the file libsonata-0.1.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
  • Upload date:
  • Size: 2.4 MB
  • Tags: CPython 3.10, manylinux: glibc 2.17+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.63.0 importlib-metadata/4.11.2 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.10

File hashes

Hashes for libsonata-0.1.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 0ab9a899163c987596f1e5b1d83125c1cc8c0929f20b5022b78627762dc86eb3
MD5 e564733c407fb3a1025c3ffea7909fce
BLAKE2b-256 32794dc1bc5d61a1259d71f4aa85323ad25debbef0eed99ca5e44292715ff89b

See more details on using hashes here.

File details

Details for the file libsonata-0.1.12-cp310-cp310-macosx_10_13_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.12-cp310-cp310-macosx_10_13_x86_64.whl
  • Upload date:
  • Size: 1.9 MB
  • Tags: CPython 3.10, macOS 10.13+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.63.0 importlib-metadata/4.11.2 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.10

File hashes

Hashes for libsonata-0.1.12-cp310-cp310-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 3625b9c05e42a9ffc4c3b5b914ffbfe6fbc6dda6caf0362305c4b3e41a4e977f
MD5 282dc0e2dd5e57ee2c5a61a1eb03f8e8
BLAKE2b-256 cb2cfda57d40b06fbf60ce5c0396c85b8300f76cbec901294922ab878b182498

See more details on using hashes here.

File details

Details for the file libsonata-0.1.12-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.12-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
  • Upload date:
  • Size: 2.4 MB
  • Tags: CPython 3.9, manylinux: glibc 2.17+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.63.0 importlib-metadata/4.11.2 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.10

File hashes

Hashes for libsonata-0.1.12-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 5c3bec1b113b28980ccdad38abd8c55779dd9186c859ba3c9a47aef35aa03837
MD5 013a61934fbe28e0bc52dd8a0c193be8
BLAKE2b-256 98bb3715c4989b28efc22d3896cbc17aa63b7525d2f23ead8fa48283e9cc0791

See more details on using hashes here.

File details

Details for the file libsonata-0.1.12-cp39-cp39-macosx_10_13_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.12-cp39-cp39-macosx_10_13_x86_64.whl
  • Upload date:
  • Size: 1.9 MB
  • Tags: CPython 3.9, macOS 10.13+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.63.0 importlib-metadata/4.11.2 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.10

File hashes

Hashes for libsonata-0.1.12-cp39-cp39-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 d76b2f30076a676cd21ac3b05feefde8aff8ac1d148513161894ece86484b8b2
MD5 dc2d341b38f15444a86a06c81c59c3ba
BLAKE2b-256 c184051360c76f9ed21a103db7d028f96c7ca7d7ff85efff417eed12e70de471

See more details on using hashes here.

File details

Details for the file libsonata-0.1.12-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.12-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
  • Upload date:
  • Size: 2.4 MB
  • Tags: CPython 3.8, manylinux: glibc 2.17+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.63.0 importlib-metadata/4.11.2 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.10

File hashes

Hashes for libsonata-0.1.12-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 24a050aff57981ed02ab222ce9c52244031d6c95c2aaff1f70d86e5bbc68b229
MD5 30503cd71a30a89db9dc71450444f4de
BLAKE2b-256 35b7ced19ccd8f1098191132e9f7797333a76913ff6442f2bc12790048958236

See more details on using hashes here.

File details

Details for the file libsonata-0.1.12-cp38-cp38-macosx_10_13_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.12-cp38-cp38-macosx_10_13_x86_64.whl
  • Upload date:
  • Size: 1.9 MB
  • Tags: CPython 3.8, macOS 10.13+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.63.0 importlib-metadata/4.11.2 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.10

File hashes

Hashes for libsonata-0.1.12-cp38-cp38-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 b3112b1c210993bf2bb5bcdfbf1771d64b704161526b0ba3fe68f6959e825632
MD5 560c8326d79c860c71b70af0451ffce2
BLAKE2b-256 9f5ad618d3bb7eae2c2039edc29495de0bb5673c181402cbb3141b0197699c76

See more details on using hashes here.

File details

Details for the file libsonata-0.1.12-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.12-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
  • Upload date:
  • Size: 2.4 MB
  • Tags: CPython 3.7m, manylinux: glibc 2.17+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.63.0 importlib-metadata/4.11.2 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.10

File hashes

Hashes for libsonata-0.1.12-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 48af69b08d7228378a27ceed3d88e97bcfeb3abc95067fbec18ee17966a51673
MD5 6a692e59b1e8b90bf7d80d2665f7f7dc
BLAKE2b-256 32ca65d2368a9431c8ad86ecf1ec1505f749b9463f3b048fd0c4b06436b7ca1c

See more details on using hashes here.

File details

Details for the file libsonata-0.1.12-cp37-cp37m-macosx_10_13_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.12-cp37-cp37m-macosx_10_13_x86_64.whl
  • Upload date:
  • Size: 1.9 MB
  • Tags: CPython 3.7m, macOS 10.13+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.63.0 importlib-metadata/4.11.2 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.10

File hashes

Hashes for libsonata-0.1.12-cp37-cp37m-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 cad13caf993480430ec16ce656cfe1fe204b40e96fbc0bc905d30ef00c2577e3
MD5 2bb36cabceb35485d2adfe4a596998e9
BLAKE2b-256 c90646bbb1a9394e404463fcb6b16dabf028ad0df3cc96154f88fb5c6aa4e768

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page