Needle Tracking Dataset - Clinical & Semi-Clinical
Authors/Creators
Description
Needle Tracking Dataset - Clinical & Semi-Clinical
Overview
This dataset contains multi-view RGB video sequences for training and evaluating markerless needle-tracking systems in CT-guided surgical environments. The data were acquired at the Research Campus STIMULATE in Magdeburg, using synchronized RGB cameras to capture realistic needle-manipulation scenarios under diverse conditions.
The dataset comprises two complementary subsets:
- Clinical Dataset: ~22,000 frames recorded in a CT laboratory replicating real interventional workflows
- Semi-Clinical Dataset: ~8,000 frames acquired under controlled laboratory conditions with systematic parameter variations
Combined, the dataset provides approximately 30,000 frames spanning a comprehensive range of needle movements, environmental conditions, and realistic procedural scenarios.
Hardware Setup
- Cameras: Three hardware-synchronized RGB cameras (1920×1080 px resolution, 60 FPS)
- Needles: Five biopsy needles with varying specifications:
- 10 cm / 14G
- 15 cm / 14G
- 15 cm / 16G
- 15 cm / 18G
- 20 cm / 16G
- Phantom: Abdomen phantom with surgical drapes
- Environment: Clinical setup mimics realistic CT interventions; semi-clinical setup focuses on controlled parameter variations
Ground Truth Acquisition
Accurate 3D needle-tip coordinates were obtained using independent reference systems operating simultaneously with the RGB-based tracking:
- Semi-Clinical Dataset: Atracsys optical tracker (Target Registration Error < 0.3 mm)
- Clinical Dataset: AprilTag-based image tracking (Target Registration Error < 0.7 mm)
All frames were processed using needle-tracking software that integrates three synchronized RGB cameras and a UNetConvNeXt segmentation model to estimate markerless tip and base positions (TRE < 0.7 mm).
Data Characteristics
Sequence Properties:
- Duration: 2 to 30 seconds per sequence
- Temporal continuity was ensured across all sequences
- Frame rate: 60 FPS
- Resolution: 1920×1080 pixels
Variation Coverage:
The dataset includes systematic variations across multiple dimensions to ensure robust model training:
1. Lighting Conditions: Range from optimal CT-lab lighting to challenging low-light scenarios
2. Occlusion Patterns: Partial occlusions from hands, surgical tools, and drapes
3. Motion Dynamics:
- Stable, controlled needle movements
- Rapid intentional displacements
- Complex manipulation patterns
- Initial stabilization phases
4. Environmental Factors:
- Different backgrounds (laboratory, CT environment)
- With and without phantom presence
- Realistic surgical drape configurations
5. User Interactions: Multiple operators with different handling techniques and viewing angles
6. Camera Perspectives: Multi-view synchronized capture from three fixed positions
Data Content
Each frame contains:
- RGB Images: Three synchronized camera views (1920×1080)
- Camera System Tracking: 2D and 3D coordinates from the tracking system
- Ground Truth: 2D and 3D coordinates from independent reference systems (Atracsys/AprilTag)
Use Cases
This dataset is designed for:
- Training and evaluating needle tracking algorithms
- Uncertainty quantification in surgical tracking systems
- Real-time needle localization in CT-guided interventions
- Robustness testing under challenging conditions (occlusion, motion, lighting)
- Multi-view triangulation and sensor fusion research
- Temporal prediction and motion modeling
- Tracking error analysis and calibration validation
Data Format
The dataset is provided as HDF5 files with JPEG-compressed images for efficient storage and fast loading.
File Structure:
needle_tracking_dataset_jpeg.h5
├── sequence_name_1/
│ ├── tracking # Camera system predictions (N_frames × 15)
│ ├── ground_truth # Reference system measurements (N_frames × 6)
│ ├── camera_0/
│ │ └── frames_jpeg # JPEG-compressed frames (variable length)
│ ├── camera_1/
│ │ └── frames_jpeg
│ └── camera_2/
│ └── frames_jpeg
├── sequence_name_2/
│ └── ...
Data Columns:
Tracking data (camera system):
- frame_id: Frame number
- tip_2d_cam1_x, tip_2d_cam1_y, tip_2d_cam2_x, tip_2d_cam2_y, tip_2d_cam3_x, tip_2d_cam3_y: 2D tip coordinates in each camera view
- base_2d_cam1_x, base_2d_cam1_y, base_2d_cam2_x, base_2d_cam2_y, base_2d_cam3_x, base_2d_cam3_y: 2D base coordinates in each camera view
- tip_3d_x, tip_3d_y, tip_3d_z: 3D tip coordinates from tracking system
Ground truth data (reference system):
- marker_tip_3d_x, marker_tip_3d_y, marker_tip_3d_z: Ground-truth 3D tip coordinates
- marker_base_3d_x, marker_base_3d_y, marker_base_3d_z: Ground-truth 3D base coordinates
Loading Example:
import h5py
from PIL import Image
import numpy as np
import io
with h5py.File('needle_tracking_dataset_clinical_jpeg.h5', 'r') as f:
seq = f['<sequence_name>']
# Load tracking data
tracking_2d = seq['tracking_2d'][:] # Shape: (N_frames, 13), dtype: int16
tracking_3d = seq['tracking_3d'][:] # Shape: (N_frames, 3), dtype: float32
# Load ground truth data
ground_truth = seq['ground_truth'][:] # Shape: (N_frames, 6), dtype: float32
# Load JPEG frame
jpeg_bytes = seq['camera_0/frames_jpeg'][0].tobytes()
img = Image.open(io.BytesIO(jpeg_bytes))
frame = np.array(img)
Compression:
- Images: JPEG quality 98 (~70-75% size reduction from PNG)
- Tracking/Ground truth: gzip compression level 5
Quality Assurance
- All sequences manually reviewed for quality
- Frames with calibration failures excluded
- Ground-truth accuracy verified against reference measurements
- Temporal consistency validated across sequences
Citation
If you use this dataset, please cite:
Steiger, M., Rezapourian, M., Rak, M., Hansen, C. (2025).
Dynamic Uncertainty Level Assessment Framework for Real-Time Needle Tracking
in CT-Guided Surgical Environments.
Funding
The work was partially funded by the German Federal Ministry of Research, Technology, and Space (within the Research Campus STIMULATE under the grant number ‘13GW0473A’) and by the European Regional Development Fund (under the operation number ‘ZS/2023/12/182010’ as part of the initiative ‘Sachsen-Anhalt WISSENSCHAFT Schwerpunkte’).
Files
Example1.png
Files
(40.2 GB)
| Name | Size | |
|---|---|---|
|
md5:62e89fd29b23f93eaa8c18d1aa5c2b06
|
2.4 MB | Preview Download |
|
md5:44ac82204cf927e26af7cbf55d28c92d
|
2.3 MB | Preview Download |
|
md5:7f81b776774888ed8c22f7cb57b189af
|
2.4 MB | Preview Download |
|
md5:5f43a70a40d52cc9ebd7b2dcd09c1e8e
|
2.3 MB | Preview Download |
|
md5:fc0404c5ddbc9d2eb6ce13747086ea2d
|
16.4 GB | Download |
|
md5:6af6addd1e4aa017a7a4a139eb2b1327
|
23.7 GB | Download |
|
md5:403321b260a4981da93d96f5a8f1ae96
|
15.4 kB | Preview Download |
Additional details
Funding
- Federal Ministry of Education and Research
- 13GW0473A