Skip to main content

SONATA files reader

Project description

banner

license coverage documentation status

libsonata

C++ / Python reader for SONATA circuit files: SONATA guide

Installation

Installing from PyPI

pip install libsonata

Installing as a Python package, directly from GitHub

pip install git+https://github.com/BlueBrain/libsonata

Building the C++ library

git clone git@github.com:BlueBrain/libsonata.git --recursive
cd libsonata
mkdir build && cd build
cmake  -DCMAKE_BUILD_TYPE=Release  -DEXTLIB_FROM_SUBMODULES=ON ..
make -j

Since libsonata uses backports for std::optional and std::variant which turn into their actual STL implementation once available, it’s recommended to compile libsonata with the same C++ standard as the project linking to libsonata. This is done by passing -DCMAKE_CXX_STANDARD={14,17} to the cmake command above.

Usage (Python)

Nodes

NodeStorage
>>> import libsonata

>>> nodes = libsonata.NodeStorage('path/to/H5/file')

# list populations
>>> nodes.population_names

# open population
>>> population = nodes.open_population(<name>)
NodePopulation
# total number of nodes in the population
>>> population.size

# attribute names
>>> population.attribute_names

# get attribute value for single node, say 42
>>> population.get_attribute('mtype', 42)

# ...or Selection of nodes (see below) => returns NumPy array with corresponding values
>>> selection = libsonata.Selection(values=[1, 5, 9, 42])  # nodes 1, 5, 9, 42
>>> mtypes = population.get_attribute('mtype', selection)
>>> list(zip(selection.flatten(), mtypes))
[(1, u'mtype_of_1'), (5, u'mtype_of_5'), (9, u'mtype_of_9'), (42, u'mtype_of_42')]
Selection

List of element IDs (either node_id, or edge_id) where adjacent IDs are grouped for the sake of efficient HDF5 file access. For instance, {1, 2, 3, 5} sequence becomes {[1, 4), [5, 6)}.

Selection can be instantiated from:
  • a sequence of scalar values (works for NumPy arrays as well)

  • a sequence of pairs (interpreted as ranges above, works for N x 2 NumPy arrays as well)

EdgePopulation connectivity queries (see below) return Selections as well.

>>> selection = libsonata.Selection([1, 2, 3, 5])
>>> selection.ranges
[(1, 4), (5, 6)]
>>> selection = libsonata.Selection([(1, 4), (5, 6)])
>>> selection.flatten()
[1, 2, 3, 5]
>>> selection.flat_size
4
>>> bool(selection)
True

Edges

EdgeStorage

Population handling for EdgeStorage is analogous to NodeStorage:

>>> edges = libsonata.EdgeStorage('path/to/H5/file')

# list populations
>>> edges.population_names

# open population
>>> population = edges.open_population(<name>)
EdgePopulation
# total number of edges in the population
>>> population.size

# attribute names
>>> population.attribute_names

# get attribute value for single edge, say 123
>>> population.get_attribute('delay', 123)

# ...or Selection of edges => returns NumPy array with corresponding values
>>> selection = libsonata.Selection([1, 5, 9])
>>> population.get_attribute('delay', selection) # returns delays for edges 1, 5, 9

…with additional methods for querying connectivity, where the results are selections that can be applied like above

# get source / target node ID for the 42nd edge:
>>> population.source_node(42)
>>> population.target_node(42)

# query connectivity (result is Selection object)
>>> selection_to_1 = population.afferent_edges(1)  # all edges with target node_id 1
>>> population.target_nodes(selection_to_1)  # since selection only contains edges
                                             # targeting node_id 1 the result will be a
                                             # numpy array of all 1's
>>> selection_from_2 = population.efferent_edges(2)  # all edges sourced from node_id 2
>>> selection = population.connecting_edges(2, 1)  # this selection is all edges from
                                                   # node_id 2 to node_id 1

# ...or their vectorized analogues
>>> selection = population.afferent_edges([1, 2, 3])
>>> selection = population.efferent_edges([1, 2, 3])
>>> selection = population.connecting_edges([1, 2, 3], [4, 5, 6])

Reports

SpikeReader
>>> import libsonata

>>> spikes = libsonata.SpikeReader('path/to/H5/file')

# list populations
>>> spikes.get_population_names()

# open population
>>> population = spikes['<name>']
SpikePopulation
# get all spikes [(node_id, timestep)]
>>> population.get()
[(5, 0.1), (2, 0.2), (3, 0.3), (2, 0.7), (3, 1.3)]

# get all spikes betwen tstart and tstop
>>> population.get(tstart=0.2, tstop=1.0)
[(2, 0.2), (3, 0.3), (2, 0.7)]

# get spikes attribute sorting (by_time, by_id, none)
>>> population.sorting
'by_time'

Pandas can be used to create a dataframe and get a better representation of the data
>>> import pandas

data = population.get()
df = pandas.DataFrame(data=data, columns=['ids', 'times']).set_index('times')
print(df)
       ids
times
0.1      5
0.2      2
0.3      3
0.7      2
1.3      3
SomaReportReader
>>> somas = libsonata.SomaReportReader('path/to/H5/file')

# list populations
>>> somas.get_population_names()

# open population
>>> population_somas = somas['<name>']
SomaReportPopulation
# get times (tstart, tstop, dt)
>>> population_somas.times
(0.0, 1.0, 0.1)

# get unit attributes
>>> population_somas.time_units
'ms'
>>> population_somas.data_units
'mV'

# node_ids sorted?
>>> population_somas.sorted
True

# get a list of all node ids in the selected population
>>> population_somas.get_node_ids()
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]

# get the DataFrame of the node_id values for the timesteps between tstart and tstop
>>> data_frame = population_somas.get(node_ids=[13, 14], tstart=0.8, tstop=1.0)

# get the data values
>>> data_frame.data
[[13.8, 14.8], [13.9, 14.9]]

# get the list of timesteps
>>> data_frame.times
[0.8, 0.9]

# get the list of node ids
>>> data_frame.ids
[13, 14]

Once again, pandas can be used to create a dataframe using the data, ids and times lists

>>> import pandas

df = pandas.DataFrame(data_frame.data, columns=data_frame.ids, index=data_frame.times)
print(df)
       13    14
0.8  13.8  14.8
0.9  13.9  14.9
ElementReportReader
>>> elements = libsonata.ElementReportReader('path/to/H5/file')

# list populations
>>> elements.get_population_names()

# open population
>>> population_elements = elements['<name>']
ElementReportPopulation
# get times (tstart, tstop, dt)
>>> population_elements.times
(0.0, 4.0, 0.2)

>>> population_elements.get_node_ids()
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]

# get the DataFrame of the node_id values for the timesteps between tstart and tstop
>>> data_frame = population_elements.get(node_ids=[13, 14], tstart=0.8, tstop=1.0)

# get the data values (list of list of floats with data[time_index][element_index])
>>> data_frame.data
[[46.0, 46.1, 46.2, 46.3, 46.4, 46.5, 46.6, 46.7, 46.8, 46.9], [56.0, 56.1, 56.2, 56.3, 56.4, 56.5, 56.6, 56.7, 56.8, 56.9]]

# get the list of timesteps
>>> data_frame.times
[0.8, 1.0]

# get the list of (node id, element_id)
>>> data_frame.ids
[(13, 30), (13, 30), (13, 31), (13, 31), (13, 32), (14, 32), (14, 33), (14, 33), (14, 34), (14, 34)]

The same way than with spikes and soma reports, pandas can be used to get a better representation of the data

>>> import pandas

df = pandas.DataFrame(data_frame.data, columns=pandas.MultiIndex.from_tuples(data_frame.ids), index=data_frame.times)
print(df)
       13                            14
       30    30    31    31    32    32    33    33    34    34
0.8  46.0  46.1  46.2  46.3  46.4  46.5  46.6  46.7  46.8  46.9
1.0  56.0  56.1  56.2  56.3  56.4  56.5  56.6  56.7  56.8  56.9

For big datasets, using numpy arrays could greatly improve the performance

>>> import numpy

np_data = numpy.asarray(data_frame.data)
np_ids = numpy.asarray(data_frame.ids).T
np_times = numpy.asarray(data_frame.times)

df = pandas.DataFrame(np_data, columns=pandas.MultiIndex.from_arrays(np_ids), index=np_times)

Acknowledgements

The development of this software was supported by funding to the Blue Brain Project, a research center of the École polytechnique fédérale de Lausanne (EPFL), from the Swiss government’s ETH Board of the Swiss Federal Institutes of Technology.

This research was supported by the EBRAINS research infrastructure, funded from the European Union’s Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 945539 (Human Brain Project SGA3). This project/research has received funding from the European Union’s Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 785907 (Human Brain Project SGA2).

License

libsonata is distributed under the terms of the GNU Lesser General Public License version 3, unless noted otherwise, for example, for external dependencies. Refer to COPYING.LESSER and COPYING files for details.

Copyright (c) 2018-2022 Blue Brain Project/EPFL

libsonata is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License version 3 as published by the Free Software Foundation.

libsonata is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with libsonata. If not, see <https://www.gnu.org/licenses/>.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

libsonata-0.1.19.tar.gz (1.0 MB view details)

Uploaded Source

Built Distributions

libsonata-0.1.19-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

libsonata-0.1.19-cp311-cp311-macosx_10_13_x86_64.whl (2.0 MB view details)

Uploaded CPython 3.11 macOS 10.13+ x86-64

libsonata-0.1.19-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

libsonata-0.1.19-cp310-cp310-macosx_10_13_x86_64.whl (2.0 MB view details)

Uploaded CPython 3.10 macOS 10.13+ x86-64

libsonata-0.1.19-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

libsonata-0.1.19-cp39-cp39-macosx_10_13_x86_64.whl (2.0 MB view details)

Uploaded CPython 3.9 macOS 10.13+ x86-64

libsonata-0.1.19-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

libsonata-0.1.19-cp38-cp38-macosx_10_13_x86_64.whl (2.0 MB view details)

Uploaded CPython 3.8 macOS 10.13+ x86-64

libsonata-0.1.19-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

libsonata-0.1.19-cp37-cp37m-macosx_10_13_x86_64.whl (2.0 MB view details)

Uploaded CPython 3.7m macOS 10.13+ x86-64

File details

Details for the file libsonata-0.1.19.tar.gz.

File metadata

  • Download URL: libsonata-0.1.19.tar.gz
  • Upload date:
  • Size: 1.0 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.16

File hashes

Hashes for libsonata-0.1.19.tar.gz
Algorithm Hash digest
SHA256 5dd7773993d00c8cb12de8adbaab85e3beb0e5e983a7333f5622075a1b09e614
MD5 e1b572555b5fe2d2df44e3078cdaa969
BLAKE2b-256 8a910aaa704f0cf5d9dbe4bed84b3066c97face85facc6614692df80141d87d9

See more details on using hashes here.

File details

Details for the file libsonata-0.1.19-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.19-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 750563fc3625d1cdc6514f80856591287c85c53181a39ff147d831e3afb63319
MD5 2a020e5e8eba7352523b277252ec9ef3
BLAKE2b-256 c531d602815e2a1bf95ce7b23978da09a476c2b900d96efe4892adef38208f6d

See more details on using hashes here.

File details

Details for the file libsonata-0.1.19-cp311-cp311-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.19-cp311-cp311-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 cfea9eea6ec68fe07475c8f34a7fb9f5604ed7ae1924e479b29b96f31133dc47
MD5 649c85ed7bd0cc868ce40ea65c8258fa
BLAKE2b-256 201a840a6e72ad7c9958da43e2690e66e8d8113fc116eb9fa77c91e91ef2ce0a

See more details on using hashes here.

File details

Details for the file libsonata-0.1.19-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.19-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 8af2635dbe71c0b7d24052c166c468fe144cc69612270073a7d77647b510296f
MD5 cf6b07cb154ec76d9f1a18b5dda515e1
BLAKE2b-256 4caad139aecd2e577fd19259ac3d29243fbcca58d0bcae15fbeffdd37d3c6bab

See more details on using hashes here.

File details

Details for the file libsonata-0.1.19-cp310-cp310-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.19-cp310-cp310-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 b1b9b99d947e72fce4eb188e063921399cecc62c384c3e02258eace69544a574
MD5 7729a06d6a2df3e86a0261bbbe10192e
BLAKE2b-256 abcf80273032e1e464f81153e800a88bcc42aa6314db28dd0a563c6182b2376c

See more details on using hashes here.

File details

Details for the file libsonata-0.1.19-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.19-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 c8c9592736f85dbee740518b0a6e6fca04c40767bb53b1c0263de8f6171a624f
MD5 261854c312c67a4abd8cea7cf8b2ebb9
BLAKE2b-256 2f1f80e17a83a7c9ab98d1e56c1e0c324424e9a98434c308867d7981e3031a8a

See more details on using hashes here.

File details

Details for the file libsonata-0.1.19-cp39-cp39-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.19-cp39-cp39-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 16aa8a48e0db9bdc6bb7cbef71a1216be6728130eb8c8fd00dd0ec8010c1be49
MD5 67fb28103180bc3781f9395ab577f7f6
BLAKE2b-256 cc90a14108d46367a66be6c468b5ba211695a974b42f9d89543926a50b25887d

See more details on using hashes here.

File details

Details for the file libsonata-0.1.19-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.19-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 1e0ac79c4267f91ee71afc2c6ae1311e057c30065f143838af54ee6bfe8af765
MD5 e718464f7745ed488b8fe57da1f3b477
BLAKE2b-256 c1bc13119afa9e302f440761e709862b0a0f829cdd6b5581840582853446445a

See more details on using hashes here.

File details

Details for the file libsonata-0.1.19-cp38-cp38-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.19-cp38-cp38-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 041ccfe1f89269e0355f41a2ea81bf4de8c38b9b971b5bd55f8450830a7def2d
MD5 4d2a8cff8f15aea781fbeeb216d5ebfc
BLAKE2b-256 b325724e404a1ba23a75e3155a8560b4051c28f9516f3e0b2380441fabe3e068

See more details on using hashes here.

File details

Details for the file libsonata-0.1.19-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.19-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 5971109f943ea1dc0145c77f7c1e38d18903d547880d32fee2661831bb5a8e4b
MD5 76ab73046a73664a567116855b3a9d5e
BLAKE2b-256 4838f0a2fc1922f6a1e3089f5d7c4fc929443f16da3edc42631713d772ed7b57

See more details on using hashes here.

File details

Details for the file libsonata-0.1.19-cp37-cp37m-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.19-cp37-cp37m-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 4e0b97348c633fc3844fba70c4888a238cdfc999b87f52383f72a7d6deab74f9
MD5 2c0c060a8ad9b60773af25011f61bc8f
BLAKE2b-256 b15f282a5c26625789ce979efffc0977c7cb28ee0ee1e4aa9628164b80f3a47b

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page