Skip to main content

SONATA files reader

Project description

banner

license coverage documentation status

libsonata

C++ / Python reader for SONATA circuit files: SONATA guide

Installation

Installing from PyPI

pip install libsonata

Installing as a Python package, directly from GitHub

pip install git+https://github.com/BlueBrain/libsonata

Building the C++ library

git clone git@github.com:BlueBrain/libsonata.git --recursive
cd libsonata
mkdir build && cd build
cmake  -DCMAKE_BUILD_TYPE=Release  -DEXTLIB_FROM_SUBMODULES=ON ..
make -j

Usage (Python)

Nodes

NodeStorage
>>> import libsonata

>>> nodes = libsonata.NodeStorage('path/to/H5/file')

# list populations
>>> nodes.population_names

# open population
>>> population = nodes.open_population(<name>)
NodePopulation
# total number of nodes in the population
>>> population.size

# attribute names
>>> population.attribute_names

# get attribute value for single node, say 42
>>> population.get_attribute('mtype', 42)

# ...or Selection of nodes (see below) => returns NumPy array with corresponding values
>>> selection = libsonata.Selection(values=[1, 5, 9, 42])  # nodes 1, 5, 9, 42
>>> mtypes = population.get_attribute('mtype', selection)
>>> list(zip(selection.flatten(), mtypes))
[(1, u'mtype_of_1'), (5, u'mtype_of_5'), (9, u'mtype_of_9'), (42, u'mtype_of_42')]
Selection

List of element IDs (either node_id, or edge_id) where adjacent IDs are grouped for the sake of efficient HDF5 file access. For instance, {1, 2, 3, 5} sequence becomes {[1, 4), [5, 6)}.

Selection can be instantiated from:
  • a sequence of scalar values (works for NumPy arrays as well)

  • a sequence of pairs (interpreted as ranges above, works for N x 2 NumPy arrays as well)

EdgePopulation connectivity queries (see below) return Selections as well.

>>> selection = libsonata.Selection([1, 2, 3, 5])
>>> selection.ranges
[(1, 4), (5, 6)]
>>> selection = libsonata.Selection([(1, 4), (5, 6)])
>>> selection.flatten()
[1, 2, 3, 5]
>>> selection.flat_size
4
>>> bool(selection)
True

Edges

EdgeStorage

Population handling for EdgeStorage is analogous to NodeStorage:

>>> edges = libsonata.EdgeStorage('path/to/H5/file')

# list populations
>>> edges.population_names

# open population
>>> population = edges.open_population(<name>)
EdgePopulation
# total number of edges in the population
>>> population.size

# attribute names
>>> population.attribute_names

# get attribute value for single edge, say 123
>>> population.get_attribute('delay', 123)

# ...or Selection of edges => returns NumPy array with corresponding values
>>> selection = libsonata.Selection([1, 5, 9])
>>> population.get_attribute('delay', selection) # returns delays for edges 1, 5, 9

…with additional methods for querying connectivity, where the results are selections that can be applied like above

# get source / target node ID for the 42nd edge:
>>> population.source_node(42)
>>> population.target_node(42)

# query connectivity (result is Selection object)
>>> selection_to_1 = population.afferent_edges(1)  # all edges with target node_id 1
>>> population.target_nodes(selection_to_1)  # since selection only contains edges
                                             # targeting node_id 1 the result will be a
                                             # numpy array of all 1's
>>> selection_from_2 = population.efferent_edges(2)  # all edges sourced from node_id 2
>>> selection = population.connecting_edges(2, 1)  # this selection is all edges from
                                                   # node_id 2 to node_id 1

# ...or their vectorized analogues
>>> selection = population.afferent_edges([1, 2, 3])
>>> selection = population.efferent_edges([1, 2, 3])
>>> selection = population.connecting_edges([1, 2, 3], [4, 5, 6])

Reports

SpikeReader
>>> import libsonata

>>> spikes = libsonata.SpikeReader('path/to/H5/file')

# list populations
>>> spikes.get_populations_names()

# open population
>>> population = spikes['<name>']
SpikePopulation
# get all spikes [(node_id, timestep)]
>>> population.get()
[(5, 0.1), (2, 0.2), (3, 0.3), (2, 0.7), (3, 1.3)]

# get all spikes betwen tstart and tstop
>>> population.get(tstart=0.2, tstop=1.0)
[(2, 0.2), (3, 0.3), (2, 0.7)]

# get spikes attribute sorting (by_time, by_id, none)
>>> population.sorting
'by_time'

Pandas can be used to create a dataframe and get a better representation of the data
>>> import pandas

data = population.get()
df = pandas.DataFrame(data=data, columns=['ids', 'times']).set_index('times')
print(df)
       ids
times
0.1      5
0.2      2
0.3      3
0.7      2
1.3      3
SomaReportReader
>>> somas = libsonata.SomaReportReader('path/to/H5/file')

# list populations
>>> somas.get_populations_names()

# open population
>>> population_somas = somas['<name>']
SomaReportPopulation
# get times (tstart, tstop, dt)
>>> population_somas.times
(0.0, 1.0, 0.1)

# get unit attributes
>>> population_somas.time_units
'ms'
>>> population_somas.data_units
'mV'

# node_ids sorted?
>>> population_somas.sorted
True

# get a list of all node ids in the selected population
>>> population_somas.get_node_ids()
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]

# get the DataFrame of the node_id values for the timesteps between tstart and tstop
>>> data_frame = population_somas.get(node_ids=[13, 14], tstart=0.8, tstop=1.0)

# get the data values
>>> data_frame.data
[[13.8, 14.8], [13.9, 14.9]]

# get the list of timesteps
>>> data_frame.times
[0.8, 0.9]

# get the list of node ids
>>> data_frame.ids
[13, 14]

Once again, pandas can be used to create a dataframe using the data, ids and times lists

>>> import pandas

df = pandas.DataFrame(data_frame.data, columns=data_frame.ids, index=data_frame.times)
print(df)
       13    14
0.8  13.8  14.8
0.9  13.9  14.9
ElementReportReader
>>> elements = libsonata.ElementReportReader('path/to/H5/file')

# list populations
>>> elements.get_populations_names()

# open population
>>> population_elements = elements['<name>']
ElementReportPopulation
# get times (tstart, tstop, dt)
>>> population_elements.times
(0.0, 4.0, 0.2)

>>> population_elements.get_node_ids()
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]

# get the DataFrame of the node_id values for the timesteps between tstart and tstop
>>> data_frame = population_elements.get(node_ids=[13, 14], tstart=0.8, tstop=1.0)

# get the data values (list of list of floats with data[time_index][element_index])
>>> data_frame.data
[[46.0, 46.1, 46.2, 46.3, 46.4, 46.5, 46.6, 46.7, 46.8, 46.9], [56.0, 56.1, 56.2, 56.3, 56.4, 56.5, 56.6, 56.7, 56.8, 56.9]]

# get the list of timesteps
>>> data_frame.times
[0.8, 1.0]

# get the list of (node id, element_id)
>>> data_frame.ids
[(13, 30), (13, 30), (13, 31), (13, 31), (13, 32), (14, 32), (14, 33), (14, 33), (14, 34), (14, 34)]

The same way than with spikes and soma reports, pandas can be used to get a better representation of the data

>>> import pandas

df = pandas.DataFrame(data_frame.data, columns=pandas.MultiIndex.from_tuples(data_frame.ids), index=data_frame.times)
print(df)
       13                            14
       30    30    31    31    32    32    33    33    34    34
0.8  46.0  46.1  46.2  46.3  46.4  46.5  46.6  46.7  46.8  46.9
1.0  56.0  56.1  56.2  56.3  56.4  56.5  56.6  56.7  56.8  56.9

For big datasets, using numpy arrays could greatly improve the performance

>>> import numpy

np_data = numpy.asarray(data_frame.data)
np_ids = numpy.asarray(data_frame.ids).T
np_times = numpy.asarray(data_frame.times)

df = pandas.DataFrame(np_data, columns=pandas.MultiIndex.from_arrays(np_ids), index=np_times)

Acknowledgements

The development of this software was supported by funding to the Blue Brain Project, a research center of the École polytechnique fédérale de Lausanne (EPFL), from the Swiss government’s ETH Board of the Swiss Federal Institutes of Technology.

This project/research has received funding from the European Union’s Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 785907 (Human Brain Project SGA2).

License

libsonata is distributed under the terms of the GNU Lesser General Public License version 3, unless noted otherwise, for example, for external dependencies. Refer to COPYING.LESSER and COPYING files for details.

Copyright (c) 2018-2022 Blue Brain Project/EPFL

libsonata is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License version 3 as published by the Free Software Foundation.

libsonata is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with libsonata. If not, see <https://www.gnu.org/licenses/>.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

libsonata-0.1.15.tar.gz (1.0 MB view details)

Uploaded Source

Built Distributions

libsonata-0.1.15-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

libsonata-0.1.15-cp311-cp311-macosx_10_13_x86_64.whl (2.0 MB view details)

Uploaded CPython 3.11 macOS 10.13+ x86-64

libsonata-0.1.15-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

libsonata-0.1.15-cp310-cp310-macosx_10_13_x86_64.whl (2.0 MB view details)

Uploaded CPython 3.10 macOS 10.13+ x86-64

libsonata-0.1.15-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

libsonata-0.1.15-cp39-cp39-macosx_10_13_x86_64.whl (2.0 MB view details)

Uploaded CPython 3.9 macOS 10.13+ x86-64

libsonata-0.1.15-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

libsonata-0.1.15-cp38-cp38-macosx_10_13_x86_64.whl (2.0 MB view details)

Uploaded CPython 3.8 macOS 10.13+ x86-64

libsonata-0.1.15-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

libsonata-0.1.15-cp37-cp37m-macosx_10_13_x86_64.whl (2.0 MB view details)

Uploaded CPython 3.7m macOS 10.13+ x86-64

File details

Details for the file libsonata-0.1.15.tar.gz.

File metadata

  • Download URL: libsonata-0.1.15.tar.gz
  • Upload date:
  • Size: 1.0 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.9.14

File hashes

Hashes for libsonata-0.1.15.tar.gz
Algorithm Hash digest
SHA256 8c7c509db692b482cba5b0453579747db5a981ce5b3c13da96b14ae0332a6e81
MD5 55efed1261b7fe5d97761e23634746ca
BLAKE2b-256 5dc4ebb26d38af4ce6ddd1eb8c46ccf33c3f348f10b2904522b7c7e421ade529

See more details on using hashes here.

File details

Details for the file libsonata-0.1.15-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.15-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 bb35961b40f3a79c24399d0ed10977084bd806d7e696a3a7b0d6d70bf30944ce
MD5 37e5c31c1af27659c1053275a409eaca
BLAKE2b-256 29b7ae84dcd3ce10cbf942cf26a6b9f5b29e59cfd4a63f3f0e8d223b5432c523

See more details on using hashes here.

File details

Details for the file libsonata-0.1.15-cp311-cp311-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.15-cp311-cp311-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 b2759a41ff3f9b8efec50524e9d78fe07cd17a35638b62a676e0de6e09627f6e
MD5 50e3b98a5356a05e8903a7173c60cb8e
BLAKE2b-256 9b4b62041cdc461ed015725a194e6cfa2ee419b187babaa1397f84273bfcf1b9

See more details on using hashes here.

File details

Details for the file libsonata-0.1.15-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.15-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 65df3df5d04363433a78408579ef04a8b3638db3f8c3cc8f7276ad895ece5373
MD5 74d99b308896740c1ecc68ddbbfbad41
BLAKE2b-256 29ea99d4c83e0c00081b9bc94b6a94471b8504c45f20410f0525ff1da7005d94

See more details on using hashes here.

File details

Details for the file libsonata-0.1.15-cp310-cp310-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.15-cp310-cp310-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 4f0b210fed87687a1eb86073a59a59e1361b64dd0afc7ef740fe6d93833e6622
MD5 4790f215cbe19b3cb8072ce83c05eb39
BLAKE2b-256 cae583ffe0a03d4b54ed8d79d7745eeaf4d62501879df98ab1942130fcb23c4a

See more details on using hashes here.

File details

Details for the file libsonata-0.1.15-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.15-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 6dc47939df50b16f077b6afc30472d266fe092949d500cfa8c60d719a10a0e90
MD5 79376bde510d65cb285507a9c2c5c375
BLAKE2b-256 6d9756f839fb0ea48ada4f530b02de248481a038d4f690bf87035865617a2d1f

See more details on using hashes here.

File details

Details for the file libsonata-0.1.15-cp39-cp39-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.15-cp39-cp39-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 def9c0c42e7cbe670f3eaa3a859bf30d12c803ebaeecf728bdd6343afa734ed3
MD5 7b2a4aba49ee367269ceb407730a7052
BLAKE2b-256 298641d2a37cc9fd30b342264de92c39e8e6efb4c5ab08385b85b3489e30eb56

See more details on using hashes here.

File details

Details for the file libsonata-0.1.15-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.15-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 920ed61abc4f1f4bef0ca67690ba24069efef898dcf21068ea650f0faa2b4bee
MD5 cfe9a5165003fc51ef5ca5d5e1c8af3a
BLAKE2b-256 ad25a8c78e51adfcbb35de6a774c9c11d9caf8fb0caaaa8675bde617590fdb0b

See more details on using hashes here.

File details

Details for the file libsonata-0.1.15-cp38-cp38-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.15-cp38-cp38-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 330036fdbac76e54fbc13d2209c3a1c2959f0f430b58c8f74ccc634fcb1d74f5
MD5 56d41d6280f53e2a8af7c378b9175e6b
BLAKE2b-256 216db7bbdf9ae1c763d12837316ebc260ba4c1646b49954e4dbd45450a551a29

See more details on using hashes here.

File details

Details for the file libsonata-0.1.15-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.15-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 cbbd7eb69d9ea2e53dddc5517732aa4075a8867de001946a22cd416ac988fa10
MD5 c8599cb7ddc983441327e13898d8540e
BLAKE2b-256 b9f93c64488de3205b813a226e1e5edc309c5e0616e12e0a71089318b627131d

See more details on using hashes here.

File details

Details for the file libsonata-0.1.15-cp37-cp37m-macosx_10_13_x86_64.whl.

File metadata

File hashes

Hashes for libsonata-0.1.15-cp37-cp37m-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 19e386c7c9ec703a108aae1cb8a4117b14a4f0a454015fef430f8feb4bbcaff3
MD5 c28587feb7866ba38637b524a31b0cf3
BLAKE2b-256 bf7b1fc2e8e775c7d57d6770655e5f1572883de96c37acfc62f08eda4f8ae6d8

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page