Skip to main content

SONATA files reader

Project description

banner

license coverage documentation status

libsonata

C++ / Python reader for SONATA circuit files: SONATA guide

Installation

Installing from PyPI

pip install libsonata

Installing as a Python package, directly from GitHub

pip install git+https://github.com/BlueBrain/libsonata

Building the C++ library

git clone git@github.com:BlueBrain/libsonata.git --recursive
cd libsonata
mkdir build && cd build
cmake  -DCMAKE_BUILD_TYPE=Release  -DEXTLIB_FROM_SUBMODULES=ON ..
make -j

Usage (Python)

Nodes

NodeStorage
>>> import libsonata

>>> nodes = libsonata.NodeStorage('path/to/H5/file')

# list populations
>>> nodes.population_names

# open population
>>> population = nodes.open_population(<name>)
NodePopulation
# total number of nodes in the population
>>> population.size

# attribute names
>>> population.attribute_names

# get attribute value for single node, say 42
>>> population.get_attribute('mtype', 42)

# ...or Selection of nodes (see below) => returns NumPy array with corresponding values
>>> selection = libsonata.Selection(values=[1, 5, 9, 42])  # nodes 1, 5, 9, 42
>>> mtypes = population.get_attribute('mtype', selection)
>>> list(zip(selection.flatten(), mtypes))
[(1, u'mtype_of_1'), (5, u'mtype_of_5'), (9, u'mtype_of_9'), (42, u'mtype_of_42')]
Selection

List of element IDs (either node_id, or edge_id) where adjacent IDs are grouped for the sake of efficient HDF5 file access. For instance, {1, 2, 3, 5} sequence becomes {[1, 4), [5, 6)}.

Selection can be instantiated from:
  • a sequence of scalar values (works for NumPy arrays as well)

  • a sequence of pairs (interpreted as ranges above, works for N x 2 NumPy arrays as well)

EdgePopulation connectivity queries (see below) return Selections as well.

>>> selection = libsonata.Selection([1, 2, 3, 5])
>>> selection.ranges
[(1, 4), (5, 6)]
>>> selection = libsonata.Selection([(1, 4), (5, 6)])
>>> selection.flatten()
[1, 2, 3, 5]
>>> selection.flat_size
4
>>> bool(selection)
True

Edges

EdgeStorage

Population handling for EdgeStorage is analogous to NodeStorage:

>>> edges = libsonata.EdgeStorage('path/to/H5/file')

# list populations
>>> edges.population_names

# open population
>>> population = edges.open_population(<name>)
EdgePopulation
# total number of edges in the population
>>> population.size

# attribute names
>>> population.attribute_names

# get attribute value for single edge, say 123
>>> population.get_attribute('delay', 123)

# ...or Selection of edges => returns NumPy array with corresponding values
>>> selection = libsonata.Selection([1, 5, 9])
>>> population.get_attribute('delay', selection) # returns delays for edges 1, 5, 9

…with additional methods for querying connectivity, where the results are selections that can be applied like above

# get source / target node ID for the 42nd edge:
>>> population.source_node(42)
>>> population.target_node(42)

# query connectivity (result is Selection object)
>>> selection_to_1 = population.afferent_edges(1)  # all edges with target node_id 1
>>> population.target_nodes(selection_to_1)  # since selection only contains edges
                                             # targeting node_id 1 the result will be a
                                             # numpy array of all 1's
>>> selection_from_2 = population.efferent_edges(2)  # all edges sourced from node_id 2
>>> selection = population.connecting_edges(2, 1)  # this selection is all edges from
                                                   # node_id 2 to node_id 1

# ...or their vectorized analogues
>>> selection = population.afferent_edges([1, 2, 3])
>>> selection = population.efferent_edges([1, 2, 3])
>>> selection = population.connecting_edges([1, 2, 3], [4, 5, 6])

Reports

SpikeReader
>>> import libsonata

>>> spikes = libsonata.SpikeReader('path/to/H5/file')

# list populations
>>> spikes.get_populations_names()

# open population
>>> population = spikes['<name>']
SpikePopulation
# get all spikes [(node_id, timestep)]
>>> population.get()
[(5, 0.1), (2, 0.2), (3, 0.3), (2, 0.7), (3, 1.3)]

# get all spikes betwen tstart and tstop
>>> population.get(tstart=0.2, tstop=1.0)
[(2, 0.2), (3, 0.3), (2, 0.7)]

# get spikes attribute sorting (by_time, by_id, none)
>>> population.sorting
'by_time'

Pandas can be used to create a dataframe and get a better representation of the data
>>> import pandas

data = population.get()
df = pandas.DataFrame(data=data, columns=['ids', 'times']).set_index('times')
print(df)
       ids
times
0.1      5
0.2      2
0.3      3
0.7      2
1.3      3
SomaReportReader
>>> somas = libsonata.SomaReportReader('path/to/H5/file')

# list populations
>>> somas.get_populations_names()

# open population
>>> population_somas = somas['<name>']
SomaReportPopulation
# get times (tstart, tstop, dt)
>>> population_somas.times
(0.0, 1.0, 0.1)

# get unit attributes
>>> population_somas.time_units
'ms'
>>> population_somas.data_units
'mV'

# node_ids sorted?
>>> population_somas.sorted
True

# get a list of all node ids in the selected population
>>> population_somas.get_node_ids()
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]

# get the DataFrame of the node_id values for the timesteps between tstart and tstop
>>> data_frame = population_somas.get(node_ids=[13, 14], tstart=0.8, tstop=1.0)

# get the data values
>>> data_frame.data
[[13.8, 14.8], [13.9, 14.9]]

# get the list of timesteps
>>> data_frame.times
[0.8, 0.9]

# get the list of node ids
>>> data_frame.ids
[13, 14]

Once again, pandas can be used to create a dataframe using the data, ids and times lists

>>> import pandas

df = pandas.DataFrame(data_frame.data, columns=data_frame.ids, index=data_frame.times)
print(df)
       13    14
0.8  13.8  14.8
0.9  13.9  14.9
ElementReportReader
>>> elements = libsonata.ElementReportReader('path/to/H5/file')

# list populations
>>> elements.get_populations_names()

# open population
>>> population_elements = elements['<name>']
ElementReportPopulation
# get times (tstart, tstop, dt)
>>> population_elements.times
(0.0, 4.0, 0.2)

>>> population_elements.get_node_ids()
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]

# get the DataFrame of the node_id values for the timesteps between tstart and tstop
>>> data_frame = population_elements.get(node_ids=[13, 14], tstart=0.8, tstop=1.0)

# get the data values (list of list of floats with data[time_index][element_index])
>>> data_frame.data
[[46.0, 46.1, 46.2, 46.3, 46.4, 46.5, 46.6, 46.7, 46.8, 46.9], [56.0, 56.1, 56.2, 56.3, 56.4, 56.5, 56.6, 56.7, 56.8, 56.9]]

# get the list of timesteps
>>> data_frame.times
[0.8, 1.0]

# get the list of (node id, element_id)
>>> data_frame.ids
[(13, 30), (13, 30), (13, 31), (13, 31), (13, 32), (14, 32), (14, 33), (14, 33), (14, 34), (14, 34)]

The same way than with spikes and soma reports, pandas can be used to get a better representation of the data

>>> import pandas

df = pandas.DataFrame(data_frame.data, columns=pandas.MultiIndex.from_tuples(data_frame.ids), index=data_frame.times)
print(df)
       13                            14
       30    30    31    31    32    32    33    33    34    34
0.8  46.0  46.1  46.2  46.3  46.4  46.5  46.6  46.7  46.8  46.9
1.0  56.0  56.1  56.2  56.3  56.4  56.5  56.6  56.7  56.8  56.9

For big datasets, using numpy arrays could greatly improve the performance

>>> import numpy

np_data = numpy.asarray(data_frame.data)
np_ids = numpy.asarray(data_frame.ids).T
np_times = numpy.asarray(data_frame.times)

df = pandas.DataFrame(np_data, columns=pandas.MultiIndex.from_arrays(np_ids), index=np_times)

Acknowledgements

The development of this software was supported by funding to the Blue Brain Project, a research center of the École polytechnique fédérale de Lausanne (EPFL), from the Swiss government’s ETH Board of the Swiss Federal Institutes of Technology.

This project/research has received funding from the European Union’s Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 785907 (Human Brain Project SGA2).

License

libsonata is distributed under the terms of the GNU Lesser General Public License version 3, unless noted otherwise, for example, for external dependencies. Refer to COPYING.LESSER and COPYING files for details.

Copyright (c) 2018-2021 Blue Brain Project/EPFL

libsonata is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License version 3 as published by the Free Software Foundation.

libsonata is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with libsonata. If not, see <https://www.gnu.org/licenses/>.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

libsonata-0.1.11.tar.gz (941.0 kB view details)

Uploaded Source

Built Distributions

libsonata-0.1.11-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

libsonata-0.1.11-cp310-cp310-macosx_10_13_x86_64.whl (1.9 MB view details)

Uploaded CPython 3.10 macOS 10.13+ x86-64

libsonata-0.1.11-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

libsonata-0.1.11-cp39-cp39-macosx_10_13_x86_64.whl (1.9 MB view details)

Uploaded CPython 3.9 macOS 10.13+ x86-64

libsonata-0.1.11-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

libsonata-0.1.11-cp38-cp38-macosx_10_13_x86_64.whl (1.9 MB view details)

Uploaded CPython 3.8 macOS 10.13+ x86-64

libsonata-0.1.11-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

libsonata-0.1.11-cp37-cp37m-macosx_10_13_x86_64.whl (1.9 MB view details)

Uploaded CPython 3.7m macOS 10.13+ x86-64

File details

Details for the file libsonata-0.1.11.tar.gz.

File metadata

  • Download URL: libsonata-0.1.11.tar.gz
  • Upload date:
  • Size: 941.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.10.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10

File hashes

Hashes for libsonata-0.1.11.tar.gz
Algorithm Hash digest
SHA256 95f302818971fec3f19ef18febd5c31c580490692138c8e4fe3534104d88b5e0
MD5 c45ba4ad953863500a1b2ad0524af19d
BLAKE2b-256 5ec0770f2a48d2171e62859fe0e07cd8bb3a4e9437ac25a6d07e5cec551be8c4

See more details on using hashes here.

File details

Details for the file libsonata-0.1.11-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.11-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
  • Upload date:
  • Size: 1.7 MB
  • Tags: CPython 3.10, manylinux: glibc 2.17+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.10.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10

File hashes

Hashes for libsonata-0.1.11-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 6af7d9d991974bc8b6b4be245a35318c62b8dd6f97f38a0f591d7cf7994748ed
MD5 009bfdce62a7ab6e4cac7b125d1932c3
BLAKE2b-256 766fe954f546126f0c95dd77beb5ff360a1de9d9e390a0c06a3779105476efdd

See more details on using hashes here.

File details

Details for the file libsonata-0.1.11-cp310-cp310-macosx_10_13_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.11-cp310-cp310-macosx_10_13_x86_64.whl
  • Upload date:
  • Size: 1.9 MB
  • Tags: CPython 3.10, macOS 10.13+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.10.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10

File hashes

Hashes for libsonata-0.1.11-cp310-cp310-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 f6ec0fa592f27636207f976555650be30d25bda0bb4b6e5c056d296cc1d3b50b
MD5 6e5328c66b6e099d06e82b072c08ebeb
BLAKE2b-256 6e6168f784b7982b8bf7148b45b8c2e9a92661ea99b6b1576c174a936253f78d

See more details on using hashes here.

File details

Details for the file libsonata-0.1.11-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.11-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
  • Upload date:
  • Size: 1.7 MB
  • Tags: CPython 3.9, manylinux: glibc 2.17+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.10.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10

File hashes

Hashes for libsonata-0.1.11-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 60a16402a8dd374836fd9c8a2cd484d45c55123b91df6e690dab9c6eb9cb6647
MD5 903ee672db0875888c910ba9520af318
BLAKE2b-256 e3d4b5abb650c406d22fa20e1cf6b40e5c2c62c51647a58c701134aae3b598ea

See more details on using hashes here.

File details

Details for the file libsonata-0.1.11-cp39-cp39-macosx_10_13_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.11-cp39-cp39-macosx_10_13_x86_64.whl
  • Upload date:
  • Size: 1.9 MB
  • Tags: CPython 3.9, macOS 10.13+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.10.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10

File hashes

Hashes for libsonata-0.1.11-cp39-cp39-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 041fc6714c3577305e5371b71c798ec28d785de3d27cc37b3f3e4a9a329be75a
MD5 305eb92f181710c4dc76e62f4518c437
BLAKE2b-256 c6ed83de14254547d54553ca5b7a3242f7557e88f26be775122fb21555e2f0c4

See more details on using hashes here.

File details

Details for the file libsonata-0.1.11-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.11-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
  • Upload date:
  • Size: 1.7 MB
  • Tags: CPython 3.8, manylinux: glibc 2.17+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.10.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10

File hashes

Hashes for libsonata-0.1.11-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 1a9bda942d524f770e7ea0dd8fb91fd71f02474d0fb97fa7e6e67c9c31d4df37
MD5 07934b2029cefa897a8ebe6aad67d274
BLAKE2b-256 87a3b9e30ae2a463ba4800102ddf4dff3fd489bd1c15228b484b9accf9eb909f

See more details on using hashes here.

File details

Details for the file libsonata-0.1.11-cp38-cp38-macosx_10_13_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.11-cp38-cp38-macosx_10_13_x86_64.whl
  • Upload date:
  • Size: 1.9 MB
  • Tags: CPython 3.8, macOS 10.13+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.10.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10

File hashes

Hashes for libsonata-0.1.11-cp38-cp38-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 a41d220f81e648efdfed09181bd5f3f8c7eec79bfb728c0b77dde11c87a8fe00
MD5 68f53f465c46994a44c0c11f2d810bc1
BLAKE2b-256 51aefc3c5ae9294fbb91248bb5aa2110e3c66c5143f0d7411011720f0c452b4d

See more details on using hashes here.

File details

Details for the file libsonata-0.1.11-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.11-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
  • Upload date:
  • Size: 1.7 MB
  • Tags: CPython 3.7m, manylinux: glibc 2.17+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.10.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10

File hashes

Hashes for libsonata-0.1.11-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 eefa4cacc79f905f3bd13474942919e634b68262c017f77f613d3ef7bd20d22e
MD5 debc793cf3439b02d31cd0ec86d70b5d
BLAKE2b-256 edc7adfea7f89fb828e152216cbfbf3a3a8e85bc705a7526895796d1b122674e

See more details on using hashes here.

File details

Details for the file libsonata-0.1.11-cp37-cp37m-macosx_10_13_x86_64.whl.

File metadata

  • Download URL: libsonata-0.1.11-cp37-cp37m-macosx_10_13_x86_64.whl
  • Upload date:
  • Size: 1.9 MB
  • Tags: CPython 3.7m, macOS 10.13+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.10.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10

File hashes

Hashes for libsonata-0.1.11-cp37-cp37m-macosx_10_13_x86_64.whl
Algorithm Hash digest
SHA256 02b83ac4a19d9c484534f4cd4163885aecbd878d40349799483cd694b3ca8ad5
MD5 91826a62a5ad926dd41d814001ac9d1d
BLAKE2b-256 2ea14889af018a4171351b4abd435d78981e58718caa888c6547f78897a46649

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page