Brain Tumor Progression Challenge

Brain tumors are among the most researched diseases in the field of medical image computing. This is reflected by the popularity of challenges like the MICCAI BraTS [1], the MICCAI FeTS challenge [2], and the use of brain MRIs in challenges like MICCAI MOOD [3] for anomaly detection.

Methods for semantic segmentation developed in this context show astonishing results, even comparable to intra- and inter-rater variability [4], with only marginal differences between the top performing participants.

However, most of the challenges and further research focus only on single time-points. Longitudinal properties are usually only inferred from the single time-point segmentation results if needed (e.g. [5]).

These longitudinal properties play a huge role in the field of brain tumor research, because they can be used for assessing treatment response. The RANO (Response Assessment in Neuro Oncology) working group [6] defines different types of response, namely complete response, partial response, stable disease and progressive disease. Progressive disease by contrast enhancing lesions can further be roughly divided into volumetric progression and instance progression. Volumetric progression is defined as an increase of the product of perpendicular diameters enhancing tumor lesions by at least 25% [6], which can roughly be translated to a gain in enhancing tumor volume above a threshold of 40% [8], instance progression is defined as the appearance of any newly formed enhancing lesion [6]. The early detection of these kinds of progression in brain tumor patients is crucial for further treatment decisions, as well as assessing the response to drugs, e.g. in clinical studies.

The two kinds of progression can be extracted in a (semi)-automatic manner from the segmentations of the individual time points.

For volumetric progression, this is a very straightforward approach, which only involves a comparison of the extracted tumor volumes and is therefore already optimized on the single scans (see [4,8]).

In the case of progression by newly formed instances the procedure is significantly more sophisticated. It involves registration between consecutive scans (for difficulties regarding longitudinal registration see [7]), the definition of "volume at risk" (i.e. volume where a new lesion might appear) and finally distinguishing newly formed lesions in this volume from lesion growth from existing lesions. A more in-depth description of this procedure is given in [8]. This is further complicated by imperfect segmentations, which might not detect the newly formed instance as they are usually not optimized for the detection of new individual lesions but instead the overall segmentation performance. Due to the ambiguities in this analysis, the procedure is prone to errors and not guaranteed to coincide with expert ratings on progression [8], even given a perfect segmentation.

The problem of semantic segmentation on single time-point scans does not fit to the detection of newly formed instances in consecutive scans. Training models directly on multiple time-points and including terms in the loss function focusing on the change of the tumor could improve the performance regarding the detection of newly formed instances. However, this still leaves the above-mentioned ambiguities in the analysis of the segmentations. An end-to-end approach for the detection of disease progression, circumventing manual interventions, would therefore be preferable.

We propose the Brain Tumor Progression Challenge (BraTPRO) to address this gap in current research. The challenge consists of two tasks, both tackling the classification of the different types of response according to RANO criteria (complete response, partial response, stable disease, progressive disease - see [6] for more details about the different types of response).

For the first task, participants will develop novel methods on our publicly available dataset with response classification annotations. They then have to submit their final method for training and evaluation on a private dataset on the organizers’ computing infrastructure. The first task represents an idealized scenario of iid hidden training and test datasets.

In the second task, participants will develop and train their method with all data available to them - including our provided public dataset with response classification annotations and any other accessible data sources. Their final model will be submitted and inference will be executed on the organizers’ hidden test set. As the hidden test set is proprietary, this represents a more realistic scenario as a shift between the participants chosen training dataset and ours may be present.

We hope that this challenge will raise awareness of the gap in current research related to longitudinal properties beyond the field of brain tumor research.

References

[1] Bakas, S. et al. "The International Brain Tumor Segmentation (BraTS) Cluster of Challenges" doi: 10.5281/zenodo.7837973
[2] Bakas, S. et al. "The Federated Tumor Segmentation (FeTS) Challenge 2022" doi: 10.5281/zenodo.6362408
[3] Zimmerer, D. et al. "Medical Out-of-Distribution Analysis Challenge 2023" doi: 10.5281/zenodo.7845019
[4] Menze, B. et al. "The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)" IEEE Transactions On Medical Imaging 34, 1993-2024 (2015)
doi: 10.1109/TMI.2014.2377694
[5] Menze, B. et al. "Proceedings of MICCAI-BRATS 2016" https://www.cbica.upenn.edu/sbia/Spyridon.Bakas/MICCAI_BraTS/MICCAI_BraTS_2016_proceedings.pdf
[6] Wen PY, Macdonald DR, Reardon DA, et al. "Updated response assessment criteria for high-grade gliomas: Response Assessment in Neuro-Oncology Working Group." Journal of Clinical Oncology 28:1963-1972 (2010)
[7] Baheti, B. et al. "The Brain Tumor Sequence Registration (BraTS-Reg) Challenge" doi: 10.5281/zenodo.6362419
[8] Kickingereder, P., Isensee, F. et al. "Automated quantitative tumor response assessment of MRI in neuro-oncology with artificial neural networks: a multicentre, retrospective study." The Lancet Oncology 20, 728-740 (2019) https://doi.org/10.1016/S1470-2045(19)30098-1