NeonRecording class#

class pyneon.NeonRecording(recording_dir: str | Path)#

Data from a single recording. The recording directory could be downloaded from either a single recording or a project on Pupil Cloud. In either case, the directory must contain an info.json file. For example, a recording directory could have the following structure:

recording_dir/
├── info.json (REQUIRED)
├── gaze.csv
├── 3d_eye_states.csv
├── imu.csv
├── blinks.csv
├── fixations.csv
├── saccades.csv
├── events.csv
├── labels.csv
├── world_timestamps.csv
├── scene_camera.json
├── <scene_video>.mp4
└── derivatives/ (PyNeon-generated derivatives)
    └── ...

Streams, events, (and scene video) will be located but not loaded until accessed as properties such as gaze, imu, and eye_states.

Parameters:

recording_dir (str or pathlib.Path) – Path to the directory containing the recording.

recording_id#

Recording ID.

Type:

str

recording_dir#

Path to the recording directory.

Type:

pathlib.Path

info#

Information about the recording. Read from info.json. For details, see https://docs.pupil-labs.com/neon/data-collection/data-format/#info-json.

Type:

dict

start_time#

Start time (in ns) of the recording as in info.json. May not match the start time of each data stream.

Type:

int

start_datetime#

Start time (datetime) of the recording as in info.json. May not match the start time of each data stream.

Type:

datetime.datetime

contents#

Contents of the recording directory. Each index is a stream or event name (e.g. gaze or imu) and columns are exist (bool), filename (str), and path (Path).

Type:

pandas.DataFrame

property gaze: NeonGaze | None#

Returns a NeonGaze object or None if no gaze data is found.

property imu: NeonIMU | None#

Returns a NeonIMU object or None if no IMU data is found.

property eye_states: NeonEyeStates | None#

Returns a NeonEyeStates object or None if no eye states data is found.

Returns a NeonBlinks object or None if no blinks data is found.

property fixations: NeonFixations | None#

Returns a NeonFixations object or None if no fixations data is found.

property saccades: NeonSaccades | None#

Returns a NeonSaccades object or None if no saccades data is found.

property events: NeonEvents | None#

Returns a NeonEvents object or None if no events data is found.

property video: NeonVideo | None#

Returns a NeonVideo object or None if no scene video is found.

concat_streams(stream_names: str | list[str], sampling_freq: Number | str = 'min', interp_float_kind: str = 'cubic', interp_other_kind: str = 'nearest', inplace: bool = False) → CustomStream#

Concatenate data from different streams under common timestamps. Since the streams may have different timestamps and sampling frequencies, resampling of all streams to a set of common timestamps is performed. The latest start timestamp and earliest last timestamp of the selected sreams are used to define the common timestamps.

Parameters:

  • stream_names (str or list of str) – Stream names to concatenate. If “all”, then all streams will be used. If a list, items must be in {"gaze", "imu", "eye_states"} ("3d_eye_states") is also tolerated as an alias for "eye_states").

  • sampling_freq (float or int or str, optional) – Sampling frequency of the concatenated streams. If numeric, the streams will be interpolated to this frequency. If "min" (default), the lowest nominal sampling frequency of the selected streams will be used. If "max", the highest nominal sampling frequency will be used.

  • interp_float_kind (str, optional) – Kind of interpolation applied on columns of float type, Defaults to "cubic". For details see scipy.interpolate.interp1d.

  • interp_other_kind (str, optional) – Kind of interpolation applied on columns of other types. Defaults to "nearest".

  • inplace (bool, optional) – Replace selected stream data with interpolated data during concatenation if``True``. Defaults to False.

Returns:

Concatenated data.

Return type:

pandas.DataFrame

concat_events(event_names: str | list[str]) → DataFrame#

Concatenate different events. All columns in the selected event type will be present in the final DataFrame. An additional "type" column denotes the event type. If events is selected, its "timestamp [ns]" column will be renamed to "start timestamp [ns]", and the "name and "type" columns will be renamed to "message name" and "message type" respectively to provide a more readable output.

Parameters:

event_names (list of str) – List of event names to concatenate. Event names must be in {"blinks", "fixations", "saccades", "events"} (singular forms are tolerated).

Returns:

Concatenated events.

Return type:

pandas.DataFrame

plot_distribution(heatmap_source: Literal['gaze', 'fixations', None] = 'gaze', scatter_source: Literal['gaze', 'fixations', None] = 'fixations', step_size: int = 10, sigma: int | float = 2, width_height: tuple[int, int] = (1600, 1200), cmap: str = 'inferno', ax: Axes | None = None, show: bool = True) → tuple[Figure, Axes]#

Plot a heatmap of gaze or fixation data on a matplotlib axis. Users can flexibly choose to generate a smoothed heatmap and/or scatter plot and the source of the data (gaze or fixation).

Parameters:

  • rec (NeonRecording) – Recording object containing the gaze and video data.

  • heatmap_source ({'gaze', 'fixations', None}) – Source of the data to plot as a heatmap. If None, no heatmap is plotted. Defaults to ‘gaze’.

  • scatter_source ({'gaze', 'fixations', None}) – Source of the data to plot as a scatter plot. If None, no scatter plot is plotted. Defaults to ‘fixations’. Gaze data is typically more dense and thus less suitable for scatter plots.

  • step_size (int) – Size of the grid cells in pixels. Defaults to 10.

  • sigma (int or float) – Standard deviation of the Gaussian kernel used to smooth the heatmap. If None or 0, no smoothing is applied. Defaults to 2.

  • width_height (tuple[int, int]) – If video is not available, the width and height of the scene camera frames to specify the heatmap dimensions. Defaults to (1600, 1200).

  • cmap (str) – Colormap to use for the heatmap. Defaults to ‘inferno’.

  • ax (matplotlib.axes.Axes or None) – Axis to plot the frame on. If None, a new figure is created. Defaults to None.

  • show (bool) – Show the figure if True. Defaults to True.

Returns:

  • fig (matplotlib.figure.Figure) – Figure object containing the plot.

  • ax (matplotlib.axes.Axes) – Axis object containing the plot.

sync_gaze_to_video(window_size: int | None = None, inplace: bool = False) → NeonGaze | None#

Synchronize gaze data to video frames by applying windowed averaging around timestamps of each video frame. See pyneon.stream.NeonStream.window_average() for details.

Parameters:

  • window_size (int, optional) – The size of the time window (in nanoseconds) over which to compute the average around each new timestamp. If None (default), the window size is set to the median interval between the new timestamps, i.e., np.median(np.diff(new_ts)). The window size must be larger than the median interval between the original data timestamps, i.e., window_size > np.median(np.diff(data.index)).

  • inplace (bool, optional) – Whether to replace the gaze data in the instance with the window averaged data.

Returns:

Gaze object containing data synchronized to video frames.

Return type:

NeonGaze

estimate_scanpath(sync_gaze: NeonGaze | None = None, lk_params: dict | None = None) → DataFrame#

Map fixations to video frames.

Parameters:

  • sync_gaze (NeonGaze) – Gaze data synchronized to video frames. If None (default), a windowed average is applied to synchronize gaze data to video frames.

  • lk_params (dict) – Parameters for the Lucas-Kanade optical flow algorithm.

Returns:

DataFrame containing the scanpath with updated fixation points.

Return type:

pandas.DataFrame

plot_scanpath_on_video(scanpath: DataFrame | None = None, circle_radius: int = 10, line_thickness: int = 2, max_fixations: int = 10, show_video: bool = False, video_output_path: Path | str = 'scanpath.mp4') → None#

Plot scanpath on top of the video frames. The resulting video can be displayed and/or saved.

Parameters:

  • scanpath (pandas.DataFrame) – DataFrame containing the fixations and gaze data.

  • circle_radius (int) – Radius of the fixation circles in pixels. Defaults to 10.

  • line_thickness (int or None) – Thickness of the lines connecting fixations. If None, no lines are drawn. Defaults to 2.

  • max_fixations (int) – Maximum number of fixations to plot per frame. Defaults to 10.

  • show_video (bool) – Whether to display the video with fixations overlaid. Defaults to False.

  • video_output_path (pathlib.Path or str or None) – Path to save the video with fixations overlaid. If None, the video is not saved. Defaults to ‘scanpath.mp4’.

export_motion_bids(motion_dir: str | Path, prefix: str = '', extra_metadata: dict = {})#

Export IMU data to Motion-BIDS format. Continuous samples are saved to a .tsv file and metadata (with template fields) are saved to a .json file. Users should later edit the metadata file according to the experiment to make it BIDS-compliant.

Parameters:

  • motion_dir (str or pathlib.Path) – Output directory to save the Motion-BIDS formatted data.

  • prefix (str, optional) – Prefix for the BIDS filenames, by default “sub-XX_task-YY_tracksys-NeonIMU”. The format should be sub-<label>[_ses-<label>]_task-<label>_tracksys-<label>[_acq-<label>][_run-<index>] (Fields in [] are optional). Files will be saved as {prefix}_motion.<tsv|json>.

  • extra_metadata (dict, optional) – Extra metadata to include in the .json file. Keys must be valid BIDS fields.

Notes

Motion-BIDS is an extension to the Brain Imaging Data Structure (BIDS) to standardize the organization of motion data for reproducible research [1]. For more information, see https://bids-specification.readthedocs.io/en/stable/modality-specific-files/motion.html.

References

export_eye_bids(output_dir: str | Path, prefix: str = '', extra_metadata: dict = {})#

Export eye-tracking data to Eye-tracking-BIDS format. Continuous samples and events are saved to .tsv.gz files with accompanying .json metadata files. Users should later edit the metadata files according to the experiment.

Parameters:

  • output_dir (str or pathlib.Path) – Output directory to save the Eye-tracking-BIDS formatted data.

  • prefix (str, optional) – Prefix for the BIDS filenames, by default “sub-XX_recording-eye”. The format should be <matches>[_recording-<label>]_<physio|physioevents>.<tsv.gz|json> (Fields in [] are optional). Files will be saved as {prefix}_physio.<tsv.gz|json> and {prefix}_physioevents.<tsv.gz|json>.

Notes

Eye-tracking-BIDS is an extension to the Brain Imaging Data Structure (BIDS) to standardize the organization of eye-tracking data for reproducible research. The extension is still being finialized. This method follows the latest standards outlined in bids-standard/bids-specification#1128.