npc_sessions

neuropixels cloud sessions

Tools for accessing data and metadata for behavior and epyhys sessions from the Mindscope Neuropixels team - in the cloud.

quickstart

Make a conda environment with python>=3.9 and simply pip install the npc_sessions package:

conda create -n npc_sessions python>=3.9
conda activate npc_sessions
pip install npc_sessions

>>> from npc_sessions import DynamicRoutingSession, get_sessions;

# each object is used to get metadata and paths for a session:         
>>> session = DynamicRoutingSession('668755_2023-08-31')  
>>> session.is_ephys                                
True
>>> session.stim_paths[0].stem                      
'DynamicRouting1_668755_20230831_131418'
# data is processed on-demand to generate individual pynwb modules:
>>> session.subject                                 # doctest: +SKIP
  subject pynwb.file.Subject at 0x1999418231888
  Fields:
    age: P205D
    age__reference: birth
    date_of_birth: 2023-02-06 20:23:02-08:00
    genotype: wt/wt
    sex: M
    species: Mus musculus
    strain: C57BL6J(NP)
    subject_id: 668755

# a full NWBFile instance can also be generated with all currently-available data:
>>> session.nwb                                     # doctest: +SKIP
root pynwb.file.NWBFile at 0x...
Fields:
  acquisition: {
    lick spout 
  }
  devices: {
    18005102491 ,
    18005114452 ,
    18005123131 ,
    18194810652 ,
    19192719021 ,
    19192719061 
  }
   ...  

# loop over all currently-tracked sessions using the session-generator:
>>> all(s.session_start_time.year >= 2022 for s in get_sessions()) # doctest: +SKIP
True
>>> trials_dfs = {}
>>> for session in get_sessions():                  # doctest: +SKIP
...     trials_dfs[session.id] = session.trials[:]

to develop with conda

To install with the intention of contributing to this package:

1. create a conda environment:

conda create -n npc_sessions python>=3.9
conda activate npc_sessions

2. clone npc_sessions from github:

git clone git@github.com:AllenInstitute/npc_sessions.git

3. pip install all dependencies:

cd npc_sessions
pip install -e .

Hierarchy of required packages

Current NWB components

key data types

• DynamicTable: for general tabular data. Cells can contain arrays (e.g. spike_times in a table of units)

  • can be accessed as a pandas dataframe

• TimeIntervals: subclass of DynamicTable which always has a start_time and stop_time column, plus other user-defined columns

• TimeSeries: has a vector array of data, a units string, and either:

  • timestamps (same length as data)
  • or starting_time and rate

• ElectricalSeries: subclass of TimeSeries with units fixed as volts

• Events: (added via an NWB extension) like TimeSeries with timestamps only, without values for data (think lick times)

all of the above also have a description, as well as other type-specific attributes

• session metadata (multiple attributes)

• subject (multiple attributes)

• devices: DynamicTable

  • physical probes (model, serial number)
  • currently only neuropixels probes

• electrode_groups: DynamicTable

  • represents the group of channels on one probe inserted in the brain
  • has session-specific info, like position relative to other probes or stereotactic coords

• electrodes: DynamicTable

  • individual channels on a probe
  • has CCF coords

• units: DynamicTable

  • metrics, links to electrodes via peak_channel

• epochs: TimeIntervals

  • start/stop time of each stim block
  • has a list of tags (includes TaskControl subclass name)

• intervals: Mapping[str, TimeIntervals]

  • 1x table per stim epoch with trials
  • behavior performance table (each block an interval)

• trials: TimeIntervals

  • same as intervals[DynamicRouting1]

• invalid_times: TimeIntervals

• acquisition: Mapping[str, Any] raw data

  • if is_ephys:
    • raw AP: Mapping[str, ElectricalSeries]
    • raw LFP: Mapping[str, ElectricalSeries]
  • if is_sync:
    • lick_sensor_rising_edges: Events
    • lick_sensor_falling_edges: Events
  • if is_task:
    • rewards: Events
  • if is_video:
    • video frame times: 1x Events per camera

• processing: Mapping[str, Any] processed/filtered data

  • behavior: Mapping[str, Any]
    • licks: Events
      • from sync or stim file
    • running_speed: TimeSeries
      • from stim file, enhanced with sync info if available
  • ecephys: Mapping[str, Any]

• analysis: Mapping[str, Any] derived data, results

  • if is_ephys:
    • all_spike_histograms: 1x TimeSeries per probe
    • drift_maps: ImageSeries
  • if is_task:
    • performance: TimeIntervals

Todo:

• filtered LFP
• stimulus templates (vis, aud, opto)
• OptogeneticStimulusSite
• analysis -> RFMaps
• per-unit response metric for each stim modality