Learn2Reg - The Challenge (2021)

Medical image registration plays a very important role in improving clinical workflows, computer-assisted interventions and diagnosis as well as for research studies involving e.g. morphological analysis. Besides ongoing research into new concepts for optimisation, similarity metrics, domain adaptation and deformation models, deep learning for medical registration is currently starting to show promising advances that could improve the robustness, generalisation, computation speed and accuracy of conventional algorithms to enable better practical translation. Nevertheless, before Learn2Reg there was no commonly used benchmark dataset to compare state-of-the-art learning based registration among another and with their conventional (not trained) counterparts. With few exceptions (CuRIOUS at MICCAI 2018/2019, the Continuous Registration Challenge at WBIR 2018 and Learn2Reg 2020) there has also been no comprehensive registration challenge covering different anatomical structures and evaluation metrics. We also believe that the entry barrier for new teams to contribute to this emerging field are higher than e.g. for segmentation, where standardised datasets (e.g. Medical Decathlon, BraTS) are easily available. In contrast, many registration tasks, require resampling from different voxel spacings, affin pre-registration and can lead to ambiguous and error-prone evaluation of whole deformation fields.

We propose a simplified challenge design that removes many of the common pitfalls for learning and applying transformations. We will provide pre-preprocessed data (resample, crop, pre-align, etc.) that can be directly employed by most conventional and learning frameworks. Only docker containers that generate displacement fields in voxel dimensions in a standard orientation will have to be provided by participants and python code to test their application (on local machines) to training data will be provided as open-source along with all evaluation metrics. Our challenge will consist of three clinically relevant sub-tasks (datasets) that are complementary in nature. They can either be individually or comprehensively addressed by participants and cover both intra- and inter-patient alignment, CT, ultrasound and MRI modalities, neuro-, thorax and abdominal anatomies and the four of the imminent challenges of medical image registration:

1. learning from small datasets
2. estimating large deformations
3. dealing with multi-modal scans
4. learning from noisy annotations

An important aspect of challenges are comprehensive and fair evaluation criteria. Since, medical image registration is not limited to accurately and robustly transferring anatomical annotations but should also provide plausible deformations, we will incorporate a measure of transformation complexity (the standard deviation of local volume change defined by the Jacobian determinant of the deformation). To encourage the submission of learning based approaches that reduce the computational burden of image registration, the run-time computation time will also be included into the ranking by awarding extra points. Due to differences in hardware and the necessity to keep some test data hidden for privacy reasons, all test submissions will be evaluated through docker containers (without releasing all the test scans). Computation time (including all steps of the employed pipeline) will be measured on identical Nvidia GPU servers the submission system, evaluation and leaderboard will be hosted on the grand-challenge.org platform (here the whole challenge and its website will also be hosted).