Skip to main content

Represent metadata for Miniscope acquisition system.

Project description

ndx-miniscope Extension for NWB

This is a Neurodata Extension (NDX) for Neurodata Without Borders (NWB) 2.0 for Miniscope acquisition data.

PyPI version

Miniscope extends the Device core NWB neurodata_type by including additional metadata for the Miniscope. Depending on the version of the acquisition software the data structure can be quite different.

Miniscope V4 format

The data recorded by the software is saved in a folder structure similar to this:

    C6-J588_Disc5/ (main folder)
    ├── 15_03_28/ (subfolder corresponding to the recording time)
    │   ├── Miniscope/ (subfolder containing the microscope video stream)
    │   │   ├── 0.avi (microscope video)
    │   │   ├── metaData.json (metadata for the microscope device)
    │   │   └── timeStamps.csv (timing of this video stream)
    │   ├── BehavCam_2/ (subfolder containing the behavioral video stream)
    │   │   ├── 0.avi (bevavioral video)
    │   │   ├── metaData.json (metadata for the behavioral camera)
    │   │   └── timeStamps.csv (timing of this video stream)
    │   └── metaData.json (metadata for the recording, such as the start time)
    ├── 15_06_28/
    │   ├── Miniscope/
    │   ├── BehavCam_2/
    │   └── metaData.json
    └── 15_12_28/

Miniscope V3 format

The Miniscope V3 acquisition software generally outputs the following files:

  • msCam[##].avi
  • behavCam[##].avi
  • timestamp.dat
  • settings_and_notes.dat

python

Installation

Get most recent release:

pip install ndx-miniscope

Install latest:

git clone https://github.com/catalystneuro/ndx-miniscope.git
cd ndx-miniscope
pip install -e .

The following code demonstrates the usage of this extension to convert Miniscope acquisition data into NWB.

Usage

from datetime import datetime
from dateutil.tz import tzlocal
import glob
import os
from pynwb import NWBFile, NWBHDF5IO
from pynwb.image import ImageSeries
from natsort import natsorted

from ndx_miniscope.utils import (
    add_miniscope_device,
    get_starting_frames,
    get_timestamps,
    read_miniscope_config,
    read_notes,
)

# The main folder that contains subfolders with the Miniscope data
folder_path = "C6-J588_Disc5/"

# Create the NWBFile
session_start_time = datetime(2017, 4, 15, 12, tzinfo=tzlocal())
nwbfile = NWBFile(
    session_description="session_description",
    identifier="identifier",
    session_start_time=session_start_time,
)

# Load the miscroscope settings
miniscope_folder_path = "C6-J588_Disc5/15_03_28/Miniscope/"
miniscope_metadata = read_miniscope_config(folder_path=miniscope_folder_path)
# Create the Miniscope device with the microscope metadata and add it to NWB
add_miniscope_device(nwbfile=nwbfile, device_metadata=miniscope_metadata)

# Load the behavioral camera settings
behavcam_folder_path = "C6-J588_Disc5/15_03_28/BehavCam_2/"
behavcam_metadata = read_miniscope_config(folder_path=behavcam_folder_path)
# Create the Miniscope device with the behavioral camera metadata and add it to NWB
add_miniscope_device(nwbfile=nwbfile, device_metadata=behavcam_metadata)

# Loading the timestamps
behavcam_timestamps = get_timestamps(folder_path=folder_path, file_pattern="BehavCam*/timeStamps.csv")
# Load the starting frames of the video files
# Note this function requires to have `cv2` installed
starting_frames = get_starting_frames(folder_path=folder_path, video_file_pattern="*/BehavCam*/*.avi")


# Legacy usage for Miniscope V3

ms_files = natsorted(glob(os.path.join(folder_path, 'msCam*.avi')))
nwbfile.add_acquisition(
    ImageSeries(
        name='OnePhotonSeries',  # this is not recommended since pynwb has native OnePhotonSeries
        format='external',
        external_file=[os.path.split(x)[1] for x in ms_files],
        timestamps=get_timestamps(folder_path=folder_path, cam_num=1),
        starting_frame=get_starting_frames(folder_path=folder_path, video_file_pattern="msCam*.avi"),
    )
)

behav_files = natsorted(glob(os.path.join(folder_path, 'behavCam*.avi')))
nwbfile.add_acquisition(
    ImageSeries(
        name='behaviorCam',
        format='external',
        external_file=[os.path.split(x)[1] for x in behav_files],
        timestamps=get_timestamps(folder_path=folder_path, cam_num=2),
        starting_frame=get_starting_frames(folder_path=folder_path, video_file_pattern="behavCam*.avi"),
    )
)

annotations = read_notes(folder_path=folder_path)
if annotations is not None:
    nwbfile.add_acquisition(annotations)


save_path = os.path.join(folder_path, "test_out.nwb")
with NWBHDF5IO(save_path, "w") as io:
    io.write(nwbfile)

# test read
with NWBHDF5IO(save_path, "r") as io:
    nwbfile_in = io.read()

MATLAB:

Installation

git clone https://github.com/bendichter/ndx-miniscope.git
generateExtension('path/to/ndx-miniscope/spec');

Usage

under construction...

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ndx-miniscope-0.5.1.tar.gz (19.2 kB view details)

Uploaded Source

Built Distribution

ndx_miniscope-0.5.1-py3-none-any.whl (11.1 kB view details)

Uploaded Python 3

File details

Details for the file ndx-miniscope-0.5.1.tar.gz.

File metadata

  • Download URL: ndx-miniscope-0.5.1.tar.gz
  • Upload date:
  • Size: 19.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.17

File hashes

Hashes for ndx-miniscope-0.5.1.tar.gz
Algorithm Hash digest
SHA256 5136d4ffdbe2c2456f7f9595aa4ceef098b280b9e4b52c5efc51713376f1d7f5
MD5 f0d135b61496518b70791faf2dda48ff
BLAKE2b-256 fa25338cb8fa6f444e5a15035fa845a9be0f40e7b9f46085b7d7ff3db39200f6

See more details on using hashes here.

File details

Details for the file ndx_miniscope-0.5.1-py3-none-any.whl.

File metadata

File hashes

Hashes for ndx_miniscope-0.5.1-py3-none-any.whl
Algorithm Hash digest
SHA256 00b82fa2501e5ba100893abfb26c48e02bc23866258c9462c61eb6f1fdf821fa
MD5 e2ef4956a5627edfea7ec6fc1c7d72b1
BLAKE2b-256 627a8ee433b95653c43bee59f7d98e2796aedfa502736a17297f28e3de7a0fbb

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page