SynthRAD2023 Challenge design: Synthesizing computed tomography for radiotherapy

Design of the SynthRAD2023 Grand Challenge https://synthrad2023.grand-challenge.org/ following the structured challenge submission provided at https://www.biomedical-challenges.org/.

The challenge will be part of MICCAI 2023.

Versions

• v0.1 design submitted on 4-11-2022 to MICCAI;
• v0.2 revised on 02-2023 as part of the rebuttal;
• v0.3 automatically updated by MICCAI: https://doi.org/10.5281/zenodo.7835406;
• v1.0 textually updated on 03-2023 according to the approved submitted changes by MICCAI (Zenodo: 10.5281/zenodo.7746020);
• v1.1 - latest textual revision + update timeline figure, added abstract and reference to MICCAI automatically generated Zenodo;

Abstract

The impact of medical imaging on oncological patients' diagnosis and therapy, has grown significantly over the last decades. Especially in radiotherapy (RT), imaging plays a crucial role in the entire workflow, from treatment simulation to patient positioning and monitoring.

Traditionally, 3D computed tomography (CT) is considered the primary imaging modality in RT, providing accurate and high-resolution patient geometry and enabling direct electron density conversion needed for dose calculations and plan optimization [Chernak et al., 1975]. X-ray-based imaging has been widely adopted for patient positioning and monitoring before, during, and after dose delivery. Recently, 3D cone-beam computed tomography (CBCT) - often integrated with the dose delivery machine - has been widely adopted, playing a vital role in traditional and more advanced image-guided adaptive radiation therapy (IGART) workflows in both photon and proton therapy. 

A key challenge in using CBCT is that several artifacts, such as shading, streaking, and cupping, affect image reconstruction. As a result, CBCT quality is insufficient to perform accurate dose calculations or replanning. Consequently, patients must be referred to a rescan CT when anatomical differences are noted between daily images and the planning CT [Ramella et al., 2017]. As an alternative, image synthesis has been proposed to improve the quality of CBCT to the CT level, producing the so-called “synthetic CT” (sCT) [Kida et al., 2018]. Additionally, CBCT-based sCT allows online adaptive CBCT-based RT workflows to improve the quality of IGART provided to the patients.

In parallel, over the last decades, magnetic resonance imaging (MRI) has also proved its added value for tumor and organs-at-risk delineation thanks to its superb soft-tissue contrast [Schmidt et al., 2015]. MRI can be acquired to match patient positioning to the planned one and monitor changes before, during, or after the dose delivery [Lagendijk et al., 2004].

To benefit from the complementary advantages offered by different imaging modalities, MRI is generally registered to CT. Such a workflow requires obtaining both CT and MRI, increasing workload, and introducing additional radiation to the patient. Recently, MRI-only based RT has been proposed to simplify and speed up the workflow, decreasing patients' exposure to ionizing radiation. This is particularly relevant for repeated simulations or fragile populations like children. MRI-only RT may reduce treatment costs and workload and eliminate residual registration errors using both imaging modalities. Additionally, MRI-only techniques can benefit MRI-guided RT [Edmund and Nyholm, 2017].

The main obstacle in introducing MRI-only RT is the lack of tissue attenuation information required for accurate dose calculations. Many methods have been proposed to convert MR to CT-equivalent images, yielding sCTs suitable for treatment planning and dose calculation. 

Artificial intelligence algorithms such as machine learning or deep learning have become the best-performing methods for deriving sCT from MRI or CBCT. With many algorithms available, all tested on different datasets, it is unclear which algorithms are better than others. Unfortunately, no public datasets or challenges have been designed to benchmark and compare different approaches. A recent review of deep learning-based sCT generation also advocated for public challenges to provide data and evaluation metrics for such open comparison [Spadea & Maspero et al., 2021].

We now designed a challenge to provide the first platform offering public datasets and evaluation metrics to benchmark and compare the latest algorithms in sCT generation. For this purpose, two tasks were defined: 1) MRI-to-sCT generation to facilitate MRI-only RT and 2) CBCT-to-sCT generation to facilitate IGART and online adaptive RT. 

A multi-center dataset of matched input (CBCT or MRI) and target (CT) image pairs with heterogeneous acquisition protocols will be divided into balanced training, validation, and test sets. We will share the inputs and targets from the training set to design and evaluate algorithms generating sCT. In the first evaluation round, called validation, input images will be shared. The participating team can upload their outputs to validate their performance and see how they rank on the leaderboard on image similarity metrics. Finally, independent test cases will be shared to generate sCT for our final evaluation. Brain and pelvic datasets from three Dutch centers (UMC Utrecht, UMC Groningen, and Radboud Nijmegen) have been collected, providing more than 500 patients for MRI-to-CT (Task 1) and 500 patients for CBCT-to-CT (Task 2) undergoing radiotherapy treatments in the corresponding departments. 

The challenge runs on https://synthrad2023.grand-challenge.org/, and the pre-processing and evaluation code used to rank the submissions based on image and dose evaluations will be shared.

Challenge participants may choose to participate only in task 1 (MRI-to-sCT), only in task 2 (CBCT-to-sCT), or in both, for both anatomical regions. In the initial training/validation phases, training and validation data will be made available to submit to an open leaderboard to enable the development of algorithms. The test input MRI/CBCT and ground truth CT will not be shared with the participants to avoid optimistic biases. 

We envision that this challenge will enable a fair and open assessment of different approaches.

References

• Chernak E.S., Rodriguez-Antunez A., Jelden G.L., Dhaliwal R.S., Lavik P.S. The use of computed tomography for radiation therapy treatment planning. Radiology. 1975 Dec;117(3):613-4. https://doi.org/10.1148/117.3.613
• Ramella, S., Fiore, M., Silipigni, S., Zappa, M. C., Jaus, M., Alberti, A. M., ... & D’Angelillo, R. M. (2017). Local control and toxicity of adaptive radiotherapy using weekly CT imaging: results from the LARTIA trial in stage III NSCLC. Journal of Thoracic Oncology, 12(7), 1122-1130. https://doi.org/10.1016/j.jtho.2017.03.025
• Kida, S., Nakamoto, T., Nakano, M., Nawa, K., Haga, A., Kotoku, J. I., ... & Nakagawa, K. (2018). Cone beam computed tomography image quality improvement using a deep convolutional neural network. Cureus, 10(4). https://doi.org/10.7759%2Fcureus.2548
• Schmidt M. A., Payne G. S. Radiotherapy planning using MRI. Phys Med Biol. 2015;60:R323 Lagendijk, J. J., Raaymakers, B. W., Van den Berg, C. A., Moerland, M. A., Philippens, M. E., & Van Vulpen, M. (2014). MR guidance in radiotherapy. Physics in Medicine & Biology, 59(21), R349. https://doi.org/10.1088/0031-9155/59/21/r349
• Edmund, J. M., & Nyholm, T. (2017). A review of substitute CT generation for MRI-only radiation therapy. Radiation Oncology, 12(1), 1-15. https://doi.org/10.1186/s13014-016-0747-y
• Spadea, M. F. & Maspero, M., Zaffino, P., & Seco, J. (2021). Deep learning-based synthetic-CT generation in radiotherapy and PET: A review. Medical Physics, 48(11), 6537–6566. https://doi.org/10.1002/mp.15150
• Schwarz, C. G., Kremers, W. K., Wiste, H. J., Gunter, J. L., Vemuri, P., Spychalla, A. J., ... & Alzheimer's Disease Neuroimaging Initiative. (2021). Changing the face of neuroimaging research: Comparing a new MRI de-facing technique with popular alternatives. NeuroImage, 231, 117845. https://doi.org/10.1016/j.neuroimage.2021.117845
• Low, D.A., Harms, W.B., Mutic, S., and Purdy, J.A. (1998), A technique for the quantitative evaluation of dose distributions. Med. Phys., 25: 656-661. https://doi.org/10.1118/1.598248
• Hall, W. A., Paulson, E., Davis, B. J., Spratt, D. E., Morgan, T. M., Dearnaley, D., ... & Lawton, C. A. (2021). NRG oncology updated international consensus atlas on pelvic lymph node volumes for intact and postoperative prostate cancer. International Journal of Radiation Oncology* Biology* Physics, 109(1), 174-185. https://doi.org/10.1016/j.ijrobp.2020.08.034
• Lambrecht, M., Eekers, D. B., Alapetite, C., Burnet, N. G., Calugaru, V., Coremans, I. E., ... & Troost, E. G. (2018). Radiation dose constraints for organs at risk in neuro-oncology; the European Particle Therapy Network consensus. Radiotherapy and Oncology, 128(1), 26-36 https://doi.org/10.1016/j.radonc.2018.05.001
• Ezzell, G. A., Burmeister, J. W., Dogan, N., LoSasso, T. J., Mechalakos, J. G., Mihailidis, D., ... & Xiao, Y. (2009). IMRT commissioning: multiple institution planning and dosimetry comparisons, a report from AAPM Task Group 119. Medical physics, 36(11), 5359-5373. https://doi.org/10.1118/1.3238104
• Drzymala, R. E., Mohan, R., Brewster, L., Chu, J., Goitein, M., Harms, W., & Urie, M. (1991). Dose-volume histograms. International Journal of Radiation Oncology* Biology* Physics, 21(1), 71-78. https://doi.org/10.1016/0360-3016(91)90168-4
• International Commission on Radiation Units and Measurements, ICRU Report 83, Prescribing, recording, and reporting intensity-modulated photon-beam therapy (IMRT)(ICRU Report 83), Bethesda, MD (2010) https://www.fnkv.cz/soubory/216/icru-83.pdf