Trackerless 3D Freehand Ultrasound Reconstruction Challenge

Reconstructing 2D Ultrasound (US) images into a 3D volume enables 3D representations of anatomy to be generated which are beneficial to a wide range of downstream tasks such as quantitative biometric measurement, multimodal registration, 3D visualisation and interventional guidance. Although substantive progress has been made recently through non-deep-learning- and deep-learning-based approaches, this application is still challenging due to 1) inherent accumulated error - frame-to-frame transformation error will be accumulated through time when reconstructing long sequence of US frames, and 2) a lack of publicly-accessible data with synchronised spatial location, often obtained from tracking devices, for benchmarking the performance and for training learning-based methods.
 
This field has witnessed the development of 3D US reconstruction from previous non-deep learning approaches such as speckle decorrelation (Chen et al. 1997) and linear regression (Prager et al. 2003) to current deep learning-based methods such as convolutional and Long Short-Term Memory (LSTM) neural networks (Guo et al. 2020, Mikaeili et al. 2022, Miura et al. 2021). One of the first deep learning based approach was proposed in (Prevost et al. 2018), in which the proposed CNN model is compared with speckle decorrelation. Since, various network models were adapted into this application, such as ConvLSTM (Luo et al. 2021) and transformers (Ning et al. 2022). Additional information (for instance signals from inertial measurement units) and sequential models have been investigated to improve the reconstruction performance (Prevost et al. 2018, Luo et al. 2022, Guo et al. 2022). However, none of these studies used the same dataset, there is a clear need for performance benchmarking to establish a standardized basis for comparison and evaluation. Moreover, several learning-based methods have relied on data from only 12-40 subjects, highlighting the necessity for additional open training data.
 
The proposed TUS-REC challenge aims to provide in vivo US data, acquired from both left and right forearms of one hundred volunteers (1200 scans, approximately 1,206,900 frames in total), tracked by a time-synchronised optical tracker, to provide a ground-truth for comparing different 3D reconstruction methods and more importantly to advance the discovery of new methods for freehand US reconstruction. The volunteer study of forearms is widely used in literature (Prevost et al. 2018, Luo et al. 2022) which is ethically and practically feasible and thus is the first step towards other specific, potentially more complex clinical applications. The outcome of the challenge includes 1) open-sourcing the first largest tracked US datasets with accurate positional information; 2) establishing one of the first benchmarks for 3D US reconstruction, suitable for modern learning-based data-driven approaches.

References

Chen et al., "Determination of scan-plane motion using speckle decorrelation: Theoretical considerations and initial test," International Journal of Imaging Systems and Technology, vol. 8, no. 1, pp. 38-44, 1997.

Guo et al., "Sensorless freehand 3D ultrasound reconstruction via deep contextual learning," In Medical Image Computing and Computer Assisted Intervention, pp. 463-472, 2020.

Guo et al., "Ultrasound volume reconstruction from freehand scans without tracking," IEEE Transactions on Biomedical Engineering, vol. 70, no. 3, pp. 970-979, 2022. 

Luo et al., "Self context and shape prior for sensorless freehand 3D ultrasound reconstruction," In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 201-210, 2021.

Luo et al., "Deep motion network for freehand 3D ultrasound reconstruction," In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 290-299, 2022.

Mikaeili et al., "Trajectory estimation of ultrasound images based on convolutional neural network," Biomedical Signal Processing and Control, vol. 78, pp. 103965, 2022.

Miura et al., "Pose estimation of 2D ultrasound probe from ultrasound image sequences using CNN and RNN," In Simplifying Medical Ultrasound: Second International Workshop, ASMUS, pp. 96-105, 2021.

Ning et al., "Spatial position estimation method for 3D ultrasound reconstruction based on hybrid transfomers," In 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1-5, 2022.

Prager et al., "Sensorless freehand 3-D ultrasound using regression of the echo intensity," Ultrasound in medicine & biology, vol. 29, no. 3, pp.437-446, 2003.

Prevost et al., "3D freehand ultrasound without external tracking using deep learning," Medical Image Analysis, vol. 48, pp. 187-202, 2018.