NeonRecording class#

class pyneon.NeonRecording(recording_dir: str | Path)#

Data from a single recording. The recording directory could be downloaded from either a single recording or a project on Pupil Cloud. In either case, the directory must contain an info.json file. For example, a recording directory could have the following structure:

recording_dir/
├── info.json (REQUIRED)
├── gaze.csv
├── 3d_eye_states.csv
├── imu.csv
├── blinks.csv
├── fixations.csv
├── saccades.csv
├── events.csv
├── labels.csv
├── world_timestamps.csv
├── scene_camera.json
├── <scene_video>.mp4 (if present)
├── scanpath.pkl (after executing `estimate_scanpath`)
└── video_with_scanpath.mp4 (after executing `overlay_scanpath_on_video`)

Streams, events, (and scene video) will be located but not loaded until accessed as properties such as gaze, imu, and eye_states.

Parameters:

recording_dir (str or pathlib.Path) – Path to the directory containing the recording.

recording_id#

Recording ID.

Type:

str

recording_dir#

Path to the recording directory.

Type:

pathlib.Path

info#

Information about the recording. Read from info.json. For details, see https://docs.pupil-labs.com/neon/data-collection/data-format/#info-json.

Type:

dict

start_time#

Start time (in ns) of the recording as in info.json. May not match the start time of each data stream.

Type:

int

start_datetime#

Start time (datetime) of the recording as in info.json. May not match the start time of each data stream.

Type:

datetime.datetime

contents#

Contents of the recording directory. Each index is a stream or event name (e.g. gaze or imu) and columns are exist (bool), filename (str), and path (Path).

Type:

pandas.DataFrame

property gaze: NeonGaze | None#

Returns a NeonGaze object or None if no gaze data is found.

property imu: NeonIMU | None#

Returns a NeonIMU object or None if no IMU data is found.

property eye_states: NeonEyeStates | None#

Returns a NeonEyeStates object or None if no eye states data is found.

Returns a NeonBlinks object or None if no blinks data is found.

property fixations: NeonFixations | None#

Returns a NeonFixations object or None if no fixations data is found.

property saccades: NeonSaccades | None#

Returns a NeonSaccades object or None if no saccades data is found.

property events: NeonEvents | None#

Returns a NeonEvents object or None if no events data is found.

property video: NeonVideo | None#

Returns a NeonVideo object or None if no scene video is found.

concat_streams(stream_names: str | list[str], sampling_freq: Number | str = 'min', resamp_float_kind: str = 'linear', resamp_other_kind: str = 'nearest', inplace: bool = False) → CustomStream#

Concatenate data from different streams under common timestamps. Since the streams may have different timestamps and sampling frequencies, resampling of all streams to a set of common timestamps is performed. The latest start timestamp and earliest last timestamp of the selected sreams are used to define the common timestamps.

Parameters:

  • stream_names (str or list of str) – Stream names to concatenate. If “all”, then all streams will be used. If a list, items must be in {"gaze", "imu", "eye_states"} ("3d_eye_states") is also tolerated as an alias for "eye_states").

  • sampling_freq (float or int or str, optional) – Sampling frequency to resample the streams to. If numeric, the streams will be resampled to this frequency. If "min", the lowest nominal sampling frequency of the selected streams will be used. If "max", the highest nominal sampling frequency will be used. Defaults to "min".

  • resamp_float_kind (str, optional) – Kind of interpolation applied on columns of float type, Defaults to "linear". For details see scipy.interpolate.interp1d.

  • resamp_other_kind (str, optional) – Kind of interpolation applied on columns of other types. Defaults to "nearest".

  • inplace (bool, optional) – Replace selected stream data with resampled data during concatenation if``True``. Defaults to False.

Returns:

concat_data – Concatenated data.

Return type:

pandas.DataFrame

concat_events(event_names: str | list[str]) → DataFrame#

Concatenate different events. All columns in the selected event type will be present in the final DataFrame. An additional "type" column denotes the event type. If events is selected, its "timestamp [ns]" column will be renamed to "start timestamp [ns]", and the "name and "type" columns will be renamed to "message name" and "message type" respectively to provide a more readable output.

Parameters:

event_names (list of str) – List of event names to concatenate. Event names must be in {"blinks", "fixations", "saccades", "events"} (singular forms are tolerated).

Returns:

concat_events – Concatenated events.

Return type:

pandas.DataFrame

plot_distribution(heatmap_source: Literal['gaze', 'fixations', None] = 'gaze', scatter_source: Literal['gaze', 'fixations', None] = 'fixations', step_size: int = 10, sigma: float | None = 2, width_height: tuple[int, int] = (1600, 1200), cmap: str | None = 'inferno', ax: Axes | None = None, show: bool = True)#

Plot a heatmap of gaze or fixation data on a matplotlib axis. Users can flexibly choose to generate a smoothed heatmap and/or scatter plot and the source of the data (gaze or fixation).

Parameters:

  • rec (NeonRecording) – Recording object containing the gaze and video data.

  • heatmap_source ({'gaze', 'fixations', None}) – Source of the data to plot as a heatmap. If None, no heatmap is plotted. Defaults to ‘gaze’.

  • scatter_source ({'gaze', 'fixations', None}) – Source of the data to plot as a scatter plot. If None, no scatter plot is plotted. Defaults to ‘fixations’. Gaze data is typically more dense and thus less suitable for scatter plots.

  • step_size (int) – Size of the grid cells in pixels. Defaults to 10.

  • sigma (float or None) – Standard deviation of the Gaussian kernel used to smooth the heatmap. If None or 0, no smoothing is applied. Defaults to 2.

  • width_height (tuple[int, int]) – If video is not available, the width and height of the scene camera frames to specify the heatmap dimensions. Defaults to (1600, 1200).

  • cmap (str or None) – Colormap to use for the heatmap. Defaults to ‘inferno’.

  • ax (matplotlib.pyplot.Axes or None) – Axis to plot the frame on. If None, a new figure is created. Defaults to None.

  • show (bool) – Show the figure if True. Defaults to True.

Returns:

  • fig (matplotlib.pyplot.Figure) – Figure object containing the plot.

  • ax (matplotlib.pyplot.Axes) – Axis object containing the plot.

sync_gaze_to_video(window_size: int | None = None) → NeonGaze#

Synchronize gaze data to video frames by applying windowed averaging around each video frame timestamp.

Parameters:#

recNeonRecording

Recording object containing gaze and video data.

window_sizeint, optional

The size of the time window (in nanoseconds) over which to compute the average around each new timestamp. If None (default), the window size is set to the median interval between the new timestamps, i.e., np.median(np.diff(new_ts)). The window size must be larger than the median interval between the original data timestamps, i.e., window_size > np.median(np.diff(data.index)).

returns:

Gaze object containing data synchronized to video frames.

rtype:

NeonGaze

estimate_scanpath(sync_gaze: NeonGaze | None = None, lk_params: dict | None = None) → DataFrame#

Map fixations to video frames.

Parameters:#

sync_gazeNeonGaze

Gaze data synchronized to video frames. If None (default), a windowed average is applied to synchronize gaze data to video frames.

lk_paramsdict

Parameters for the Lucas-Kanade optical flow algorithm.

plot_scanpath_on_video(scanpath: DataFrame, circle_radius: int = 10, line_thickness: int | None = 2, max_fixations: int = 10, show_video: bool = False, video_output_path: Path | str | None = 'scanpath.mp4') → None#

Plot scanpath on top of the video frames. The resulting video can be displayed and/or saved.

Parameters:

  • scanpath (pandas.DataFrame) – DataFrame containing the fixations and gaze data.

  • circle_radius (int) – Radius of the fixation circles in pixels. Defaults to 10.

  • line_thickness (int or None) – Thickness of the lines connecting fixations. If None, no lines are drawn. Defaults to 2.

  • max_fixations (int) – Maximum number of fixations to plot per frame. Defaults to 10.

  • show_video (bool) – Whether to display the video with fixations overlaid. Defaults to False.

  • video_output_path (pathlib.Path or str or None) – Path to save the video with fixations overlaid. If None, the video is not saved. Defaults to ‘scanpath.mp4’.

export_motion_bids(motion_dir: str | Path, prefix: str = '', extra_metadata: dict = {})#

Export IMU data to Motion-BIDS format. Continuous samples are saved to a .tsv file and metadata (with template fields) are saved to a .json file. Users should later edit the metadata file according to the experiment to make it BIDS-compliant.

Parameters:

  • motion_dir (str or pathlib.Path) – Output directory to save the Motion-BIDS formatted data.

  • prefix (str, optional) – Prefix for the BIDS filenames, by default “sub-XX_task-YY_tracksys-NeonIMU”. The format should be sub-<label>[_ses-<label>]_task-<label>_tracksys-<label>[_acq-<label>][_run-<index>] (Fields in [] are optional). Files will be saved as {prefix}_motion.<tsv|json>.

Notes

Motion-BIDS is an extension to the Brain Imaging Data Structure (BIDS) to standardize the organization of motion data for reproducible research [1]. For more information, see https://bids-specification.readthedocs.io/en/stable/modality-specific-files/motion.html.

References

export_eye_bids(output_dir: str | Path, prefix: str = '', extra_metadata: dict = {})#

Export eye-tracking data to Eye-tracking-BIDS format. Continuous samples and events are saved to .tsv.gz files with accompanying .json metadata files. Users should later edit the metadata files according to the experiment.

Parameters:

  • output_dir (str or pathlib.Path) – Output directory to save the Eye-tracking-BIDS formatted data.

  • prefix (str, optional) – Prefix for the BIDS filenames, by default “sub-XX_recording-eye”. The format should be <matches>[_recording-<label>]_<physio|physioevents>.<tsv.gz|json> (Fields in [] are optional). Files will be saved as {prefix}_physio.<tsv.gz|json> and {prefix}_physioevents.<tsv.gz|json>.

Notes

Eye-tracking-BIDS is an extension to the Brain Imaging Data Structure (BIDS) to standardize the organization of eye-tracking data for reproducible research. The extension is still being finialized. This method follows the latest standards outlined in bids-standard/bids-specification#1128.