STIR Challenge 2024 Dataset

This is the STIR Challenge 2024 (STIRC2024) dataset for evaluating tracking and reconstruction methods. This a new, previously unreleased, dataset that was used in the STIR Challenge 2024 at MICCAI's EndoVis. This data has the same format as the larger STIR training and validation dataset (STIROrig). Dataset info for STIRC2024 is in the 2024 Challenge arxiv paper 
The STIR training and validation dataset (STIROrig) details are available in the dataset curation and creation arxiv paper. A download link for STIROrig can be found here: https://dx.doi.org/10.21227/w8g4-g548

Challenge Paper Link:

https://arxiv.org/abs/2503.24306

Data Description

This dataset includes multiple collections (folders).  Each collection includes multiple clips with stereo videos.
Under each of 'left' and 'right' folders are:
seq** folders (seq00-seq14, for example). Each 'seq' folder corresponds to an action.
For example, left/seq01 corresponds to the same recording as right/seq01 for the right camera. All images and videos are stereo rectified, and camera calibration is also provided in `calib.json`.

Each 'seq**' folder contains:
- frames (a folder containing visible light video as a mp4)
- segmentation (contains start and end binary segmentation pngs)
- icgstartseg.png (start segmentation)
- icgendseg.png (finish segmentation)
- <STARTTIME>ms_icgstart.png (starting IR image)
- <STARTTIME>ms_icgend.png (finishing IR image)

Quantification

Use the STIRMetrics repo to assist with quantification, we use the TAP-Vid metric of average accuracy, δx, over thresholds. The 2D metric is averaged over accuracy thresholds of [4, 8, 16, 32, 64] pixels. The 3D metric is averaged over accuracy thresholds of [2, 4, 8, 16, 32] millimetres.

Refer to the challenge paper for additional analysis.

Terms

By using this dataset, you agree to cite the 2024 challenge paper:

@misc{schmidtpointtrackingsurgerythe2024,
      title={Point Tracking in Surgery--The 2024 Surgical Tattoos in Infrared (STIR) Challenge}, 
      author={Adam Schmidt and Mert Asim Karaoglu and Soham Sinha and Mingang Jang and Ho-Gun Ha and Kyungmin Jung and Kyeongmo Gu and Ihsan Ullah and Hyunki Lee and Jonáš Šerých and Michal Neoral and Jiří Matas and Rulin Zhou and Wenlong He and An Wang and Hongliang Ren and Bruno Silva and Sandro Queirós and Estêvão Lima and João L. Vilaça and Shunsuke Kikuchi and Atsushi Kouno and Hiroki Matsuzaki and Tongtong Li and Yulu Chen and Ling Li and Xiang Ma and Xiaojian Li and Mona Sheikh Zeinoddin and Xu Wang and Zafer Tandogdu and Greg Shaw and Evangelos Mazomenos and Danail Stoyanov and Yuxin Chen and Zijian Wu and Alexander Ladikos and Simon DiMaio and Septimiu E. Salcudean and Omid Mohareri},
      year={2025},
      eprint={2503.24306},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2503.24306}, 
}

And the initial dataset paper:

@article{schmidt2024surgical,
  title={Surgical Tattoos in Infrared: A Dataset for Quantifying Tissue Tracking and Mapping},
  author={Schmidt, Adam and Mohareri, Omid and DiMaio, Simon and Salcudean, Septimiu E},
  journal={IEEE Transactions on Medical Imaging},
  year={2024},
  publisher={IEEE}
}