{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exporting to BIDS formats\n",
    "\n",
    "\n",
    "The [Brain Imaging Data Structure (BIDS)](https://bids.neuroimaging.io/index.html) is a comprehensive framework designed to systematically organize and share diverse types of data, including behavioral, physiological, and neuroimaging information. Converting datasets into BIDS format is a widely adopted methodology, particularly in the process of curating datasets that adhere to the principles of FAIR (Findable, Accessible, Interoperable, Reusable).\n",
    "\n",
    "For datasets encompassing mobile eye-tracking data, it is essential to apply specific BIDS specifications tailored for such data. In this context, Motion-BIDS and Eye-Tracking-BIDS specifications are noteworthy. Motion-BIDS ([BEP029](https://github.com/bids-standard/bids-specification/pull/981)) has been successfully integrated into the official BIDS specification, demonstrating its readiness for use in organizing motion-related data. On the other hand, Eye-Tracking-BIDS ([BEP020](https://github.com/bids-standard/bids-specification/pull/1128)) is still undergoing development, reflecting ongoing efforts to provide a standardized format for eye-tracking data. You can find more information about these specifications in the following references:\n",
    "\n",
    "> <cite>Gorgolewski, K., Auer, T., Calhoun, V. et al. The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments. Sci Data 3, 160044 (2016). https://doi.org/10.1038/sdata.2016.44<cite>\n",
    "\n",
    "> <cite>Jeung, S., Cockx, H., Appelhoff, S. et al. Motion-BIDS: an extension to the brain imaging data structure to organize motion data for reproducible research. Sci Data 11, 716 (2024). https://doi.org/10.1038/s41597-024-03559-8<cite>\n",
    "\n",
    "In the ensuing section, we will delve into the procedure for exporting data to Motion-BIDS. This will be accomplished using the `export_motion_bids` method available within PyNeon's `Recording` objects, offering a practical guide for researchers aiming to standardize their motion data in alignment with the BIDS framework."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyneon import get_sample_data, Recording\n",
    "import pandas as pd\n",
    "import json\n",
    "\n",
    "rec_dir = (\n",
    "    get_sample_data(\"screenFlash\")\n",
    "    / \"Timeseries Data + Scene Video\"\n",
    "    / \"screenflash-54b2f924\"\n",
    ")\n",
    "rec = Recording(rec_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To use `export_motion_bids` method, we need to specify the output directory where the BIDS dataset will be saved, and a string prefix to denote this session of data. The prefix follows the following format (Fields in [] are optional):\n",
    "\n",
    "```text\n",
    "sub-<label>[_ses-<label>]_task-<label>_tracksys-<label>[_acq-<label>][_run-<index>]\n",
    "```\n",
    "\n",
    "If you have any additional metadata that you would like to include, you can pass it as a dictionary to the `extra_metadata` argument. This metadata will be saved in the `dataset_description.json` file.\n",
    "\n",
    "Let's see what files will be exported to the BIDS dataset directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sub-1_task-screenFlash_tracksys-NeonIMU_run-1_channels.json\n",
      "sub-1_task-screenFlash_tracksys-NeonIMU_run-1_channels.tsv\n",
      "sub-1_task-screenFlash_tracksys-NeonIMU_run-1_motion.json\n",
      "sub-1_task-screenFlash_tracksys-NeonIMU_run-1_motion.tsv\n"
     ]
    }
   ],
   "source": [
    "# Create a BIDS directory\n",
    "motion_dir = rec_dir.parent / \"BIDS\" / \"sub-1\" / \"motion\"\n",
    "motion_dir.mkdir(exist_ok=True, parents=True)\n",
    "\n",
    "# Export the motion data to BIDS format\n",
    "prefix = \"sub-1_task-screenFlash_tracksys-NeonIMU_run-1\"\n",
    "extra_metadata = {\n",
    "    \"TaskName\": \"screenFlash\",\n",
    "    \"InstitutionName\": \"Streeling University\",\n",
    "    \"InstitutionAddress\": \"Trantor, Galactic Empire\",\n",
    "    \"InstitutionalDepartmentName\": \"Department of Psychohistory\",\n",
    "}\n",
    "\n",
    "rec.export_motion_bids(motion_dir, prefix=prefix, extra_metadata=extra_metadata)\n",
    "\n",
    "# Print all the conents of motion_dir\n",
    "for path in motion_dir.iterdir():\n",
    "    print(path.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The contents of these files follow the Motion-BIDS specification at: https://bids-specification.readthedocs.io/en/stable/modality-specific-files/motion.html.\n",
    "\n",
    "For example, the `_motion.tsv` is a tab-separated values file that contains the (n_samples, n_channels) motion data without a header:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Motion data shape: (4944, 13)\n",
      "   -0.032425  1.249313  -1.062393  -0.027344  -0.378906  0.949707  \\\n",
      "0   0.022678  1.190656  -1.119273  -0.019153  -0.377996  0.940606   \n",
      "1   0.291978  1.451646  -1.228879  -0.024460  -0.373712  0.939519   \n",
      "2   0.139505  1.396314  -1.294069  -0.030831  -0.371104  0.944893   \n",
      "3   0.399156  1.061873  -1.306534  -0.030095  -0.371316  0.943091   \n",
      "4   0.307300  1.362720  -1.348332  -0.027290  -0.373589  0.944235   \n",
      "\n",
      "   1.1495382546655957  -22.014470833356658  131.97212848840843  \\\n",
      "0            1.154530           -22.012730          131.961423   \n",
      "1            1.161212           -22.009442          131.950870   \n",
      "2            1.171125           -22.004620          131.941577   \n",
      "3            1.178265           -21.998308          131.931277   \n",
      "4            1.183449           -21.993049          131.918883   \n",
      "\n",
      "   0.4012015163898468  -0.0866925716400146  -0.1703909933567047  \\\n",
      "0            0.401294            -0.086741            -0.170352   \n",
      "1            0.401388            -0.086798            -0.170295   \n",
      "2            0.401478            -0.086873            -0.170215   \n",
      "3            0.401573            -0.086922            -0.170133   \n",
      "4            0.401681            -0.086962            -0.170064   \n",
      "\n",
      "   0.895817220211029  \n",
      "0           0.895779  \n",
      "1           0.895742  \n",
      "2           0.895709  \n",
      "3           0.895678  \n",
      "4           0.895638  \n"
     ]
    }
   ],
   "source": [
    "motion_tsv_path = motion_dir / f\"{prefix}_motion.tsv\"\n",
    "motion_df = pd.read_csv(motion_tsv_path, sep=\"\\t\")\n",
    "print(f\"Motion data shape: {motion_df.shape}\")\n",
    "print(motion_df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Its metadata is stored in the `_motion.json` file, which contains (note the extra metadata we added):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"TaskName\": \"screenFlash\",\n",
      "  \"TaskDescription\": \"\",\n",
      "  \"Instructions\": \"\",\n",
      "  \"DeviceSerialNumber\": \"321970\",\n",
      "  \"Manufacturer\": \"TDK InvenSense & Pupil Labs\",\n",
      "  \"ManufacturersModelName\": \"ICM-20948\",\n",
      "  \"SoftwareVersions\": \"App version: 2.9.3-prod; Pipeline version: 2.8.0\",\n",
      "  \"InstitutionName\": \"Streeling University\",\n",
      "  \"InstitutionAddress\": \"Trantor, Galactic Empire\",\n",
      "  \"InstitutionalDepartmentName\": \"Department of Psychohistory\",\n",
      "  \"SamplingFrequency\": 110,\n",
      "  \"ACCELChannelCount\": 3,\n",
      "  \"GYROChannelCount\": 3,\n",
      "  \"MissingValues\": \"n/a\",\n",
      "  \"MotionChannelCount\": 13,\n",
      "  \"ORNTChannelCount\": 7,\n",
      "  \"SubjectArtefactDescription\": \"\",\n",
      "  \"TrackedPointsCount\": 0,\n",
      "  \"TrackingSystemName\": \"IMU included in Neon\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "motion_json = motion_dir / f\"{prefix}_motion.json\"\n",
    "with open(motion_json, \"r\") as f:\n",
    "    motion_json_data = json.load(f)\n",
    "print(json.dumps(motion_json_data, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The metadata for each channel (each column in the `_motion.tsv` file) is stored in `_channels.tsv` file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              name component   type tracked_point      units  \\\n",
      "0           gyro x         x   GYRO          Head      deg/s   \n",
      "1           gyro y         y   GYRO          Head      deg/s   \n",
      "2           gyro z         z   GYRO          Head      deg/s   \n",
      "3   acceleration x         x  ACCEL          Head          g   \n",
      "4   acceleration y         y  ACCEL          Head          g   \n",
      "5   acceleration z         z  ACCEL          Head          g   \n",
      "6             roll         x   ORNT          Head        deg   \n",
      "7            pitch         y   ORNT          Head        deg   \n",
      "8              yaw         z   ORNT          Head        deg   \n",
      "9     quaternion w         w   ORNT          Head  arbitrary   \n",
      "10    quaternion x         x   ORNT          Head  arbitrary   \n",
      "11    quaternion y         y   ORNT          Head  arbitrary   \n",
      "12    quaternion z         z   ORNT          Head  arbitrary   \n",
      "\n",
      "    sampling_frequency  \n",
      "0                  110  \n",
      "1                  110  \n",
      "2                  110  \n",
      "3                  110  \n",
      "4                  110  \n",
      "5                  110  \n",
      "6                  110  \n",
      "7                  110  \n",
      "8                  110  \n",
      "9                  110  \n",
      "10                 110  \n",
      "11                 110  \n",
      "12                 110  \n"
     ]
    }
   ],
   "source": [
    "channels_tsv_path = motion_dir / f\"{prefix}_channels.tsv\"\n",
    "channels_df = pd.read_csv(channels_tsv_path, sep=\"\\t\")\n",
    "print(channels_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, the `_channels.json` file contains the coordinate system:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"reference_frame\": {\n",
      "    \"Levels\": {\n",
      "      \"global\": {\n",
      "        \"SpatialAxes\": \"RAS\",\n",
      "        \"RotationOrder\": \"ZXY\",\n",
      "        \"RotationRule\": \"right-hand\",\n",
      "        \"Description\": \"This global reference frame is defined by the IMU axes: X right, Y anterior, Z superior. The scene camera frame differs from this frame by a 102-degree rotation around the X-axis. All motion data are expressed relative to the IMU frame for consistency.\"\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "channels_json_path = motion_dir / f\"{prefix}_channels.json\"\n",
    "with open(channels_json_path, \"r\") as f:\n",
    "    channels_json_data = json.load(f)\n",
    "print(json.dumps(channels_json_data, indent=2))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "pyneon",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}