[TULIP-Bench Sample] Multi-perspective videos and 3D poses capturing multi-task Parkinson's disease assessments and deep brain stimulation treatment effects: Representative Data Sample

Notice for Authorized Reviewers — Private Access Link
This private access link has been generated solely for the purpose of reviewing this academic submission to NeurIPS 2026. The record is publicly accessible, but files are restricted to users with access. You are receiving private access links to enable review of the dataset for academic purposes only. Only access the dataset if you (1) agree to use the dataset only for the purpose of reviewing this academic product, and (2) accept the following license agreement. By downloading any file from this repository using this link, you agree to the following conditions, which constitute a legally binding data use agreement :

• You will use the data exclusively for the purpose of reviewing this academic submission and for no other purpose.
• You will not redistribute, share, publish, or transmit this link or any downloaded files to any third party.
• You will delete all downloaded files upon completion of the review process.
• You will not attempt to re-identify any individuals in the dataset.
• You will not use the data for commercial purposes.
• This dataset, and any copyrights therein, are owned by Duke University. You agree to an open license under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BYNC-ND 4.0) license: https://creativecommons.org/licenses/by-nc-nd/4.0/. Outside contributions to the Duke-owned dataset cannot be accepted unless the contributor assigns copyrights to any modifications, changes, and/or derivatives over to Duke University. Please note that this dataset is distributed AS IS, WITHOUT ANY WARRANTY; and without the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Downloading any file constitutes your acceptance of these terms.

Note on file access: This repository uses Zenodo's restricted-access system to comply with our data use agreement. You may see the message "The record is publicly accessible, but files are restricted to users with access" when visiting the page; this is expected behavior and does not mean your access has been denied. If you have received a private access link, you can download all files directly using that link without creating a Zenodo account or logging in. If you encounter persistent access issues, please contact us at kyungdo.kim@duke.edu.

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

This repository provides a representative sample of the TULIP-Bench dataset, released to allow reviewers and researchers to inspect data quality, structure, and formats prior to requesting full access. This sample includes one representative subject per activity type, covering all data modalities released in the full dataset
(Fist_Left: Neurips_Sub3, Fist_Right: Neurips_Sub5, Gait_normalPD: Neurips_Sub46, Gait_Neurostimulation: Neurips_DBS_Sub10)

TULIP-Bench is the first publicly available PD movement dataset to jointly provide (1) raw multi-perspective synchronized video, (2) triangulated markerless 3D pose ground truth derived from six hardware-synchronized cameras rather than monocular estimation, (3) both fine motor (bilateral hand movement) and gross motor (gait) tasks in the same cohort, and (4) within-patient treatment recordings with deep brain stimulation (DBS) toggled OFF and ON. No prior public dataset provides all four simultaneously (see Table 1 in the accompanying paper). TULIP-Bench's triangulated multi-camera ground truth enables honest evaluation of monocular and lifting-based 3D pose methods, including the finding that off-the-shelf monocular methods (WHAM, SAM 3D Body).

• Recording setup: Six hardware-synchronized Basler cameras (acA1920-155uc, 1920x1200, 80 fps). Layout: Camera 5 (front-facing), Camera 4 (ceiling bird's-eye), Cameras 1, 2, 3, 6 (four elevated corner views). For gait, only Cameras 1, 2, 3, 6 are released; Camera 1 is the designated monocular baseline view. All 6 cameras released for fist activities. Calibrated using OpenCV with a charucoboard pattern.

[Sample Data Structure]
* Camera_parameters/
   All projection matrices (P = K @ [R|t], shape 3x4) can be used directly for multi-view triangulation and 2D reprojection onto the original video frames.

*Pose3D_Fist_Left_sample_subjects/
    sample_subject_pose3d_lefthand.pkl   — Left-hand 3D pose (2400×21×3, 30s x 80fps, real-world mm scale)
*Pose3D_Fist_Right_sample_subjects/
    sample_subject_pose3d_righthand.pkl  — Right-hand 3D pose (2400×21×3, 30s x 80fps,real-world mm scale)
*Pose3D_Gait_normalPD_sample_subject/
    sample_subject_pose3d_body.pkl       — Gait 3D body pose (7200×33×3, 90s x 80fps, real-world mm scale)
*Pose3D_Gait_Neurostimulation_DBS_subject/
    sample_subject_OFF_pose3d_body.pkl   — DBS OFF gait pose (7200×33×3, 90s x 80fps, real-world mm scale)
    sample_subject_ON_pose3d_body.pkl    — DBS ON gait pose (7200×33×3, 90s x 80fps, real-world mm scale)

*Video_Fist_Left_sample_subjects/
    Camera1.mp4 ... Camera6.mp4          — Face-blurred videos from 6 cameras
*Video_Fist_Right_sample_subjects/
    Camera1.mp4 ... Camera6.mp4          — Face-blurred videos from 6 cameras
*Video_Gait_normal_PD_subject/
    Camera1.mp4, Camera2.mp4,
    Camera3.mp4, Camera6.mp4             — Face-blurred gait videos, 4 cameras
*Video_Gait_Neurostimulation_DBS_subject/
        OFF_state/
             Camera1.mp4, Camera2.mp4,
             Camera3.mp4, Camera6.mp4             — Face-blurred DBS gait videos, 4 cameras
        ON_state/
             Camera1.mp4, Camera2.mp4,
             Camera3.mp4, Camera6.mp4             — Face-blurred DBS gait videos, 4 cameras
    (covers both neurostimulation OFF- and ON- states)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Full dataset (https://doi.org/10.5281/zenodo.20027113) contains:

TULIP-Bench is a multi-camera Parkinson's disease (PD) movement dataset combining synchronized RGB videos and triangulated 3D pose sequences across bilateral fist closure (hand movement, MDS-UPDRS Item 3.5) and gait (MDS-UPDRS Item 3.10) activities, released alongside the paper "Multi-Perspective Video Analysis for Parkinson's Disease Severity Prediction and Treatment Response Detection" (submitted to NeurIPS 2026 Evaluations and Datasets Track).

• Cohorts
  • Observational cohort (normal_PD, N=56 PD patients): Multi-camera recordings of bilateral fist closure (hand movement) and gait at self-selected pace. Each activity has an independent MDS-UPDRS score rated by a licensed movement disorder specialist.
  • Neurostimulation (Deep Brain Stimulation, DBS) cohort (N=10 subjects, N=20 recordings total for DBS-ON/OFF): Separately recruited patients recorded performing gait in both stimulation OFF (after 10-minute washout) and stimulation ON (after confirmed stable clinical effect) states. Includes per-state UPDRS Gait scores and structured body-part severity annotations (arm swing L/R, stride L/R, elbow flexion L/R).

• Dataset structure
  • Fist_Video.zip: Face-blurred MP4 videos, FistL & FistR, 6 cameras, 56 subjects; 30 seconds with 80fps
  • Fist_Pose_3D.zip: Triangulated 3D hand poses (2400x21x3, mm; MediaPipe Hand 21-keypoint convention, recommend excluding first 3s to remove hand-raising phase)
  • Fist_Camera_parameters.zip: Per-subject 3x4 projection matrices (P = K @ [R|t]), Camera 1-6
  • Fist_labels.csv: MDS-UPDRS Item 3.5 scores (UPDRS_Fist_L, UPDRS_Fist_R)
  • Gait_Videos.zip: Face-blurred MP4 videos, Gait for normal PD (56 subjects), & Gait_DBS (10 subjects with N=20 recordings total for ON/OFF), 4 cameras-Cameras 1, 2, 3, 6; 90 seconds with 80fps
  • Gait_Pose_3D.zip: Triangulated 3D body poses (7200x33x3, mm; 17-keypoint MediaPipe subset per paper)
  • Gait_Camera_parameters.zip: Per-subject 3x4 projection matrices, all 6 cameras stored (Cameras 4 and 5 included in pkl but videos not released for gait)
  • Gait_normalPD_labels.csv: MDS-UPDRS gait scores, for observational cohort
  • Gait_DBS_labels.csv: MDS-UPDRS and body-part severity labels, for DBS cohort

Access is restricted to academic researchers. Please submit a request via the form below and provide your institutional affiliation and intended use.

IRB approval: Pro00112856 and Pro00116644.

All participants provided written informed consent.