Preprocessing module#
- pyneon.preprocess.interpolate(new_ts: ndarray, data: DataFrame, float_kind: str = 'linear', other_kind: str = 'nearest') DataFrame #
Interpolate a data stream to a new set of timestamps.
- Parameters:
new_ts (numpy.ndarray) – An array of new timestamps (in nanoseconds) at which to evaluate the interpolant.
data (pandas.DataFrame) – Source data to interpolate. Must have a monotonically increasing index named
timestamp [ns]
.float_kind (str, optional) – Kind of interpolation applied on columns of
float
type, For details seescipy.interpolate.interp1d
. Defaults to"linear"
.other_kind (str, optional) – Kind of interpolation applied on columns of other types, For details see
scipy.interpolate.interp1d
. Defaults to"nearest"
.
- Returns:
Interpolated data with the same columns and dtypes as
data
and indexed bynew_ts
.- Return type:
- pyneon.preprocess.interpolate_events(data: DataFrame, events: Events, buffer: Number | tuple[Number, Number] = 0.05, float_kind: str = 'linear', other_kind: str = 'nearest') DataFrame #
Interpolate data in the duration of events in the stream data. Similar to
mne.preprocessing.eyetracking.interpolate_blinks()
.- Parameters:
data (pandas.DataFrame) – Data to interpolate. Must have a monotonically increasing index named
timestamp [ns]
.events (Events) – Events object containing the events to interpolate. The events must have
start timestamp [ns]
andend timestamp [ns]
columns.buffer (numbers.Number or , optional) – The time before and after an event (in seconds) to consider invalid. If a single number is provided, the same buffer is applied to both before and after the event. Defaults to 0.05.
float_kind (str, optional) – Kind of interpolation applied on columns of
float
type, For details seescipy.interpolate.interp1d
. Defaults to"linear"
.other_kind (str, optional) – Kind of interpolation applied on columns of other types, For details see
scipy.interpolate.interp1d
. Defaults to"nearest"
.
- Returns:
Interpolated data with the same columns and dtypes as
data
and indexed bydata.index
.- Return type:
- pyneon.preprocess.window_average(new_ts: ndarray, data: DataFrame, window_size: int | None = None) DataFrame #
Take the average over a time window to obtain smoothed data at new timestamps.
- Parameters:
new_ts (numpy.ndarray) –
An array of new timestamps (in nanoseconds) at which to evaluate the averaged signal. Must be coarser than the source sampling, i.e.:
>>> np.median(np.diff(new_ts)) > np.median(np.diff(data.index))
data (pandas.DataFrame) – Source data to apply window average to. Must have a monotonically increasing index named
timestamp [ns]
.window_size (int, optional) – The size of the time window (in nanoseconds) over which to compute the average around each new timestamp. If
None
(default), the window size is set to the median interval between the new timestamps, i.e.,np.median(np.diff(new_ts))
. The window size must be larger than the median interval between the original data timestamps, i.e.,window_size > np.median(np.diff(data.index))
.
- Returns:
Data with window average applied, carrying the same columns and dtypes as
data
and indexed bynew_ts
. Non-float columns are rounded back to their original integer type after averaging.- Return type:
- pyneon.preprocess.concat_streams(rec: Recording, stream_names: str | list[str] = 'all', sampling_freq: Number | str = 'min', interp_float_kind: str = 'linear', interp_other_kind: str = 'nearest', inplace: bool = False) DataFrame #
Concatenate data from different streams under common timestamps. Since the streams may have different timestamps and sampling frequencies, interpolation of all streams to a set of common timestamps is performed. The latest start timestamp and earliest last timestamp of the selected streams are used to define the common timestamps.
- Parameters:
rec (Recording) – Recording object containing the streams to concatenate.
stream_names (str or list of str) – Stream names to concatenate. If “all” (default), then all streams will be used. If a list, items must be in
{"gaze", "imu", "eye_states"}
("3d_eye_states"
) is also tolerated as an alias for"eye_states"
).sampling_freq (numbers.Number or str, optional) – Sampling frequency of the concatenated streams. If numeric, the streams will be interpolated to this frequency. If “min” (default), the lowest nominal sampling frequency of the selected streams will be used. If “max”, the highest nominal sampling frequency will be used.
interp_float_kind (str, optional) – Kind of interpolation applied on columns of
float
type, Defaults to"linear"
. For details seescipy.interpolate.interp1d
.interp_other_kind (str, optional) – Kind of interpolation applied on columns of other types. Defaults to
"nearest"
.inplace (bool, optional) – Replace selected stream data with interpolated data during concatenation if``True``. Defaults to
False
.
- Returns:
concat_data – Concatenated data.
- Return type:
- pyneon.preprocess.concat_events(rec: Recording, event_names: str | list[str]) DataFrame #
Concatenate different events. All columns in the selected event type will be present in the final DataFrame. An additional
type
column denotes the event type. If"events"
is inevent_names
, itstimestamp [ns]
column will be renamed tostart timestamp [ns]
, and thename
andtype
columns will be renamed tomessage name
andmessage type
respectively to prevent confusion between physiological events and user-supplied messages.- Parameters:
- Returns:
Concatenated events.
- Return type:
- pyneon.preprocess.smooth_camera_pose(camera_position_raw: DataFrame, initial_state_noise: float = 0.1, process_noise: float = 0.1, measurement_noise: float = 0.01, gating_threshold: float = 2.0, bidirectional: bool = False) DataFrame #
Apply a Kalman filter to smooth camera positions, with optional forward-backward smoothing (RTS smoother). Handles missing measurements and propagates predictions.
- Parameters:
camera_position_raw (pandas.DataFrame) – DataFrame containing ‘frame_idx’ and ‘camera_pos’ columns.
initial_state_noise (float, optional) – Initial state covariance scaling factor. Default is 0.1.
process_noise (float, optional) – Process noise covariance scaling factor. Default is 0.1.
measurement_noise (float, optional) – Measurement noise covariance scaling factor. Default is 0.01.
gating_threshold (float, optional) – Mahalanobis distance threshold for gating outliers. Default is 2.0.
bidirectional (bool, optional) – If True, applies forward-backward RTS smoothing. Default is False.
- Returns:
A DataFrame with ‘frame_idx’ and ‘smoothed_camera_pos’.
- Return type:
pd.DataFrame