Preprocessing module#

pyneon.preprocess.interpolate(new_ts: ndarray, data: DataFrame, float_kind: str = 'cubic', other_kind: str = 'nearest') → DataFrame#

Interpolate a data stream to a new set of timestamps.

Parameters:

  • new_ts (numpy.ndarray) – An array of new timestamps (in nanoseconds) at which to evaluate the interpolant.

  • data (pandas.DataFrame) – Data to interpolate. Must have a monotonically increasing index named timestamp [ns].

  • float_kind (str, optional) – Kind of interpolation applied on columns of float type, by default "cubic". For details see scipy.interpolate.interp1d.

  • other_kind (str, optional) – Kind of interpolation applied on columns of other types, by default "nearest". For details see scipy.interpolate.interp1d.

Returns:

Interpolated data.

Return type:

pandas.DataFrame

pyneon.preprocess.window_average(new_ts: ndarray, data: DataFrame, window_size: int | None = None) → DataFrame#

Take the average over a time window to obtain smoothed data at new timestamps.

Parameters:

  • new_ts (numpy.ndarray) – An array of new timestamps (in nanoseconds) at which to compute the windowed averages. The median interval between these new timestamps must be larger than the median interval between the original data timestamps, i.e., np.median(np.diff(new_ts)) > np.median(np.diff(data.index)). In other words, only downsampling is supported.

  • data (pandas.DataFrame) – Data to apply window average to. Must have a monotonically increasing index named timestamp [ns].

  • window_size (int, optional) – The size of the time window (in nanoseconds) over which to compute the average around each new timestamp. If None (default), the window size is set to the median interval between the new timestamps, i.e., np.median(np.diff(new_ts)). The window size must be larger than the median interval between the original data timestamps, i.e., window_size > np.median(np.diff(data.index)).

Returns:

Data with window average applied.

Return type:

pandas.DataFrame

pyneon.preprocess.concat_streams(rec: NeonRecording, stream_names: str | list[str] = 'all', sampling_freq: Number | str = 'min', interp_float_kind: str = 'cubic', interp_other_kind: str = 'nearest', inplace: bool = False) → DataFrame#

Concatenate data from different streams under common timestamps. Since the streams may have different timestamps and sampling frequencies, interpolation of all streams to a set of common timestamps is performed. The latest start timestamp and earliest last timestamp of the selected streams are used to define the common timestamps.

Parameters:

  • rec (NeonRecording) – NeonRecording object containing the streams to concatenate.

  • stream_names (str or list of str) – Stream names to concatenate. If “all”, then all streams will be used. If a list, items must be in {"gaze", "imu", "eye_states"} ("3d_eye_states") is also tolerated as an alias for "eye_states").

  • sampling_freq (float or int or str, optional) – Sampling frequency of the concatenated streams. If numeric, the streams will be interpolated to this frequency. If "min" (default), the lowest nominal sampling frequency of the selected streams will be used. If "max", the highest nominal sampling frequency will be used.

  • interp_float_kind (str, optional) – Kind of interpolation applied on columns of float type, Defaults to "cubic". For details see scipy.interpolate.interp1d.

  • interp_other_kind (str, optional) – Kind of interpolation applied on columns of other types. Defaults to "nearest".

  • inplace (bool, optional) – Replace selected stream data with interpolated data during concatenation if``True``. Defaults to False.

Returns:

concat_data – Concatenated data.

Return type:

pandas.DataFrame

pyneon.preprocess.concat_events(rec: NeonRecording, event_names: str | list[str]) → DataFrame#

Concatenate different events. All columns in the selected event type will be present in the final DataFrame. An additional type column denotes the event type. If "events" is in event_names, its timestamp [ns] column will be renamed to start timestamp [ns], and the name and type columns will be renamed to message name and message type respectively to prevent confusion between physiological events and user-supplied messages.

Parameters:

  • rec (NeonRecording) – NeonRecording object containing the events to concatenate.

  • event_names (list of str) – List of event names to concatenate. Event names must be in {"blinks", "fixations", "saccades", "events"} (singular forms are tolerated).

Returns:

Concatenated events.

Return type:

pandas.DataFrame