TULIP Dataset (CVPR 2024): Multi-Camera Videos and Clinician Ratings of the MDS-UPDRS Part III Motor Exam for Parkinson's Disease Assessment
Authors/Creators
-
1.
Duke University
-
2.
Duke University School of Medicine
Description
TULIP Dataset (Version 1.0.3)
** Updates! We include one more subject, so total we have 12 subjects in this dataset.
Overview:
The TULIP (Three-dimensional Understanding and Learning of Impairments in Parkinson’s) dataset provides high-resolution RGB data from multi-camera setups, supporting research on precision motor assessments for Parkinson’s Disease (PD). Version 1.0.0 features synchronized RGB data from six cameras, capturing multiple angles of PD and healthy participants performing clinically relevant motor tasks. We chose the name TULIP, a nod to the floral emblem of PD research and advocacy, to symbolize our goal for this dataset, to foster transformative new machine learning approaches for PD understanding and treatment.
Data Contents:
RGB videos are organized by subject ID and activity, with synchronized recordings from six camera perspectives per activity. Each video documents motor tasks such as gait and finger tapping, in line with the UPDRS standards for PD, allowing for a detailed study of joint angles, tremors, and other movements key to understanding PD progression.
· Multi-Camera Video Data: RGB videos from six cameras enable robust 3D pose extraction.
· Metadata: Includes camera parameters (intrinsic and extrinsic matrices for 3D reconstruction) and task descriptions.
· Clinical Examination Labels: Task-specific labels aligned with clinical motor assessments, with annotations from three clinicians to aid in automated scoring models. We also provided the labels in the csv file format.
File Structure:
Data is organized by Subject ID > Activities > Camera Perspective, with each folder containing RGB video files. Annotations and activity labels are available in a CSV file for easy correlation of tasks with motor patterns. For the camera parameters (pickle file), each subject has its own set of parameters. When you open the pickle file, the order of the elements is as follows: [proj_matrices, cam_matrices, extrinsic_matrices, rmatrices, rvecs, tvecs, distcoeffs]. Here’s a brief description of each:
· proj_matrices: Camera projection matrix (3x4 format)
· cam_matrices: Intrinsic camera matrix
· extrinsic_matrices: Extrinsic camera matrix
· rmatrices: Rotation matrix
· rvecs: Rotation vector
· tvecs: Translation vector
· distcoeffs: Distortion coefficients, which is a zero matrix in our case.
Privacy and Consent:
Faces are blurred to ensure privacy. This initial version of the dataset contains data from 12 participants, with data from the remaining 3 participants expected to be released in the near future.
Code Availability:
Behavioral feature extraction demo code for 3D poses will be available on our github (github link can be found on the TULIP Project page). For further details and access to our publication on TULIP data and baseline projects, please visit TULIP Project or CVPR 2024 Paper.
Citing this dataset:
Please cite our CVPR paper if use this dataset in your work. Citation:
Kyungdo Kim, Sihan Lyu, Sneha Mantri, Timothy W. Dunn; TULIP: Multi-camera 3D Precision Assessment of Parkinson's Disease; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 22551-22562
Files
Restricted
The record is publicly accessible, but files are restricted. <a href="https://zenodo.org/account/settings/login?next=https://zenodo.org/records/14983978">Log in</a> to check if you have access.
Additional details
Related works
- Is published in
- Conference paper: https://openaccess.thecvf.com/content/CVPR2024/html/Kim_TULIP_Multi-camera_3D_Precision_Assessment_of_Parkinsons_Disease_CVPR_2024_paper.html (URL)