Extension for storing large-scale simulation output in the Neurodata Without Borders: Neurophysiology format
Project description
# nwbext_simulation_output: An extension for output data of large-scale simulations
Developed in collaboration between the Soltesz lab and the Allen Institute during [NWB Hackathon #4](https://github.com/NeurodataWithoutBorders/nwb_hackathons/tree/master/HCK04_2018_Seattle/Projects/NetworkOutput) by Ben Dichter*, Kael Dai*, Aaron Milstein, Yazan Billeh, Andrew Tritt, Jean-Christophe Fillion-Robin, Anton Akhipov, Oliver Ruebel, Nicholas Cain, Kristofer Bouchard, and Ivan Soltesz
This extension defines a single NWB data type, `CompartmentSeries`, that allows you to store continuous data (e.g. membrane potential) from many compartments of many cells in a scalable way.
![Image of CompartmentSeries](docs/source/_static/multicompartment_schema_1.png)
This structure stores an arbitrarily large number of cells and cellular compartments with 5 datasets. It can scale to a million or more neurons, and enables efficient parallel read and write. It is designed to handle NEURON output data and to easily interface with the SONATA format.
## Guide
### python
#### installation
```
pip install git+https://github.com/bendichter/simulation_output.git
```
#### usage
```python
from pynwb import NWBHDF5IO, NWBFile
from datetime import datetime
from nwbext_simulation_output import CompartmentSeries, Compartments
import numpy as np
compartments = Compartments()
compartments.add_row(number=[0, 1, 2, 3, 4], position=[0.1, 0.2, 0.3, 0.4, 0.5])
compartments.add_row(number=[0], position=[np.nan])
cs = CompartmentSeries('membrane_potential', np.random.randn(10, 6),
compartments=compartments,
unit='V', rate=100.)
nwbfile = NWBFile('description', 'id', datetime.now().astimezone())
nwbfile.add_acquisition(compartments)
nwbfile.add_acquisition(cs)
with NWBHDF5IO('test_compartment_series.nwb', 'w') as io:
io.write(nwbfile)
```
### MATLAB
#### installation
command line:
```
git clone https://github.com/bendichter/nwbext_simulation_output.git
```
in matlab:
```matlab
generateExtension('/path/to/nwbext_simulation_output/nwbext_simulation_output/nwbext_simulation_output.namespace.yaml');
```
#### usage
```matlab
[number, number_index] = util.create_indexed_column( ...
{[0, 1, 2, 3, 4], 0}, '/acquisition/compartments/number');
[position, position_index] = util.create_indexed_column( ...
{[0.1, 0.2, 0.3, 0.4, 0.5], 0}, '/acquisition/compartments/position');
compartments = types.simulation_output.Compartments( ...
'colnames', {'number', 'position'}, ...
'description', 'membrane potential from various compartments', ...
'id', types.core.ElementIdentifiers('data', int64(0:5)));
compartments.position = position;
compartments.position_index = position_index;
compartments.number = number;
compartments.number_index = number_index;
membrane_potential = types.simulation_output.CompartmentSeries( ...
'data', randn(10,6), ...
'compartments', types.untyped.SoftLink('/acquisition/compartments'), ...
'data_unit', 'V', ...
'starting_time_rate', 100., ...
'starting_time', 0.0);
nwb.acquisition.set('compartments', compartments);
nwb.acquisition.set('membrane_potential', membrane_potential);
```
## Talks
Ben Dichter*, Kael Dai*, Aaron Milstein, Yazan Billeh, Andrew Tritt, Jean-Christophe Fillion-Robin, Anton Akhipov, Oliver Ruebel, Nicholas Cain, Kristofer Bouchard, Ivan Soltesz. NWB extension for storing results of large-scale neural network simulations. NeuroInformatics. Montreal, Canada (2018). [video](https://www.youtube.com/watch?v=uuYQW0EE2GY).
Developed in collaboration between the Soltesz lab and the Allen Institute during [NWB Hackathon #4](https://github.com/NeurodataWithoutBorders/nwb_hackathons/tree/master/HCK04_2018_Seattle/Projects/NetworkOutput) by Ben Dichter*, Kael Dai*, Aaron Milstein, Yazan Billeh, Andrew Tritt, Jean-Christophe Fillion-Robin, Anton Akhipov, Oliver Ruebel, Nicholas Cain, Kristofer Bouchard, and Ivan Soltesz
This extension defines a single NWB data type, `CompartmentSeries`, that allows you to store continuous data (e.g. membrane potential) from many compartments of many cells in a scalable way.
![Image of CompartmentSeries](docs/source/_static/multicompartment_schema_1.png)
This structure stores an arbitrarily large number of cells and cellular compartments with 5 datasets. It can scale to a million or more neurons, and enables efficient parallel read and write. It is designed to handle NEURON output data and to easily interface with the SONATA format.
## Guide
### python
#### installation
```
pip install git+https://github.com/bendichter/simulation_output.git
```
#### usage
```python
from pynwb import NWBHDF5IO, NWBFile
from datetime import datetime
from nwbext_simulation_output import CompartmentSeries, Compartments
import numpy as np
compartments = Compartments()
compartments.add_row(number=[0, 1, 2, 3, 4], position=[0.1, 0.2, 0.3, 0.4, 0.5])
compartments.add_row(number=[0], position=[np.nan])
cs = CompartmentSeries('membrane_potential', np.random.randn(10, 6),
compartments=compartments,
unit='V', rate=100.)
nwbfile = NWBFile('description', 'id', datetime.now().astimezone())
nwbfile.add_acquisition(compartments)
nwbfile.add_acquisition(cs)
with NWBHDF5IO('test_compartment_series.nwb', 'w') as io:
io.write(nwbfile)
```
### MATLAB
#### installation
command line:
```
git clone https://github.com/bendichter/nwbext_simulation_output.git
```
in matlab:
```matlab
generateExtension('/path/to/nwbext_simulation_output/nwbext_simulation_output/nwbext_simulation_output.namespace.yaml');
```
#### usage
```matlab
[number, number_index] = util.create_indexed_column( ...
{[0, 1, 2, 3, 4], 0}, '/acquisition/compartments/number');
[position, position_index] = util.create_indexed_column( ...
{[0.1, 0.2, 0.3, 0.4, 0.5], 0}, '/acquisition/compartments/position');
compartments = types.simulation_output.Compartments( ...
'colnames', {'number', 'position'}, ...
'description', 'membrane potential from various compartments', ...
'id', types.core.ElementIdentifiers('data', int64(0:5)));
compartments.position = position;
compartments.position_index = position_index;
compartments.number = number;
compartments.number_index = number_index;
membrane_potential = types.simulation_output.CompartmentSeries( ...
'data', randn(10,6), ...
'compartments', types.untyped.SoftLink('/acquisition/compartments'), ...
'data_unit', 'V', ...
'starting_time_rate', 100., ...
'starting_time', 0.0);
nwb.acquisition.set('compartments', compartments);
nwb.acquisition.set('membrane_potential', membrane_potential);
```
## Talks
Ben Dichter*, Kael Dai*, Aaron Milstein, Yazan Billeh, Andrew Tritt, Jean-Christophe Fillion-Robin, Anton Akhipov, Oliver Ruebel, Nicholas Cain, Kristofer Bouchard, Ivan Soltesz. NWB extension for storing results of large-scale neural network simulations. NeuroInformatics. Montreal, Canada (2018). [video](https://www.youtube.com/watch?v=uuYQW0EE2GY).
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file nwbext_simulation_output-0.1.tar.gz
.
File metadata
- Download URL: nwbext_simulation_output-0.1.tar.gz
- Upload date:
- Size: 5.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 04e27b1f4b96d123a19a38667cd82da78ee57ec379c6bee61dc11f2bb61ff466 |
|
MD5 | 946c15ae1b4673142588ab4e0a03456f |
|
BLAKE2b-256 | cc1ab465d98ce7f21bb4d825aa49a5a04854a973bca3ed5506162b383f841891 |
File details
Details for the file nwbext_simulation_output-0.1-py3-none-any.whl
.
File metadata
- Download URL: nwbext_simulation_output-0.1-py3-none-any.whl
- Upload date:
- Size: 6.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6bb1dfa836e6ae45ae7040177b1b1a170ee2e6ba9449e512d8d81c5eadff26a3 |
|
MD5 | 88c5b7ab67399047a510106b2f661e31 |
|
BLAKE2b-256 | f1735c50223e1e02f54cc74a13effe5974cf4d515afedc303edf838ca42416a9 |