Visualization module#

pyneon.vis.plot_frame(video: NeonVideo, index: int = 0, ax: Axes | None = None, auto_title: bool = True, show: bool = True) → tuple[Figure, Axes]#

Plot a frame from the video on a matplotlib axis.

Parameters:

  • video (NeonVideo) – Video object to plot the frame from.

  • index (int) – Index of the frame to plot.

  • ax (matplotlib.axes.Axes or None) – Axis to plot the frame on. If None, a new figure is created. Defaults to None.

  • auto_title (bool) – Whether to automatically set the title of the axis. The automatic title includes the video file name and the frame index. Defaults to True.

Returns:

  • fig (matplotlib.figure.Figure) – Figure object containing the plot.

  • ax (matplotlib.axes.Axes) – Axis object containing the plot.

pyneon.vis.plot_distribution(rec: NeonRecording, heatmap_source: Literal['gaze', 'fixations', None] = 'gaze', scatter_source: Literal['gaze', 'fixations', None] = 'fixations', step_size: int = 10, sigma: int | float = 2, width_height: tuple[int, int] = (1600, 1200), cmap: str = 'inferno', ax: Axes | None = None, show: bool = True) → tuple[Figure, Axes]#

Plot a heatmap of gaze or fixation data on a matplotlib axis. Users can flexibly choose to generate a smoothed heatmap and/or scatter plot and the source of the data (gaze or fixation).

Parameters:

  • rec (NeonRecording) – Recording object containing the gaze and video data.

  • heatmap_source ({'gaze', 'fixations', None}) – Source of the data to plot as a heatmap. If None, no heatmap is plotted. Defaults to ‘gaze’.

  • scatter_source ({'gaze', 'fixations', None}) – Source of the data to plot as a scatter plot. If None, no scatter plot is plotted. Defaults to ‘fixations’. Gaze data is typically more dense and thus less suitable for scatter plots.

  • step_size (int) – Size of the grid cells in pixels. Defaults to 10.

  • sigma (int or float) – Standard deviation of the Gaussian kernel used to smooth the heatmap. If None or 0, no smoothing is applied. Defaults to 2.

  • width_height (tuple[int, int]) – If video is not available, the width and height of the scene camera frames to specify the heatmap dimensions. Defaults to (1600, 1200).

  • cmap (str) – Colormap to use for the heatmap. Defaults to ‘inferno’.

  • ax (matplotlib.axes.Axes or None) – Axis to plot the frame on. If None, a new figure is created. Defaults to None.

  • show (bool) – Show the figure if True. Defaults to True.

Returns:

  • fig (matplotlib.figure.Figure) – Figure object containing the plot.

  • ax (matplotlib.axes.Axes) – Axis object containing the plot.

pyneon.vis.overlay_scanpath(video: NeonVideo, scanpath: DataFrame, circle_radius: int = 10, line_thickness: int = 2, max_fixations: int = 10, show_video: bool = False, video_output_path: Path | str | None = 'scanpath.mp4') → None#

Plot scanpath on top of the video frames. The resulting video can be displayed and/or saved.

Parameters:

  • video (NeonVideo) – Video object to overlay the fixations on.

  • scanpath (pandas.DataFrame) – DataFrame containing the fixations and gaze data.

  • circle_radius (int) – Radius of the fixation circles in pixels. Defaults to 10.

  • line_thickness (int or None) – Thickness of the lines connecting fixations. If None, no lines are drawn. Defaults to 2.

  • max_fixations (int) – Maximum number of fixations to plot per frame. Defaults to 10.

  • show_video (bool) – Whether to display the video with fixations overlaid. Defaults to False.

  • video_output_path (pathlib.Path or str or None) – Path to save the video with fixations overlaid. If None, the video is not saved. Defaults to ‘scanpath.mp4’.

pyneon.vis.overlay_detections_and_pose(recording: NeonRecording, april_detections: DataFrame, camera_positions: DataFrame, room_corners: ndarray = array([[0, 0], [0, 1], [1, 1], [1, 0]]), video_output_path: Path | str = 'detection_and_pose.mp4', graph_size: ndarray = array([300, 300]), show_video: bool = True)#

Overlay AprilTag detections and camera positions on top of the recording’s video frames. Additionally, draw an inset plot showing the camera’s trajectory within known environmental (room) boundaries.

This function reads frames from the provided recording video and overlays: - AprilTag detections (if present in the current frame) - The camera position, using a mini-map inset showing all previously visited positions,

as well as the current/last known position of the camera.

The mini-map inset uses predetermined room boundaries derived from the provided room_corners array to map positions onto a fixed coordinate system, allowing consistent visualization of the camera’s movement over time.

Parameters:

  • recording (NeonRecording) – Recording object containing the video and related metadata.

  • april_detections (pandas.DataFrame) –

    DataFrame containing AprilTag detections for each frame, with columns:
    • ’frame_idx’: int

      The frame number.

    • ’tag_id’: int

      The ID of the detected AprilTag.

    • ’corners’: np.ndarray of shape (4,2)

      Pixel coordinates of the tag’s corners.

  • camera_positions (pandas.DataFrame) –

    DataFrame containing the camera positions for each frame, with at least:
    • ’frame_idx’: int

      The frame number.

    • ’smoothed_camera_pos’: np.ndarray of shape (3,)

      The camera position [x, y, z] in world coordinates.

  • room_corners (np.ndarray of shape (N, 2), optional) – Array defining the polygon corners of the room in world coordinates. Defaults to a simple unit square: [[0,0],[0,1],[1,1],[1,0]].

  • video_output_path (str or pathlib.Path, optional) – Path to save the output video with overlays. Defaults to ‘output_with_overlays.mp4’.

  • graph_size (np.ndarray of shape (2,), optional) – The width and height (in pixels) of the inset mini-map. Defaults to [300, 300].

  • show_video (bool, optional) – Whether to display the video with overlays as it is processed. Press ‘ESC’ to stop early. Defaults to True.

Returns:

The function saves the processed video to the specified output path and optionally displays it during processing. No value is returned.

Return type:

None

Notes

  • If the video cannot be read, a RuntimeError is raised.

  • Press ‘ESC’ to stop playback if show_video is True.