Deep Image Generation Model Challenge in Surgery 2022

The continuing AdaptOR Challenge aims to spark methodological developments in deep image generation models for the surgical domain. In this year's edition, we again focus on video-assisted mitral valve repair [1], which is becoming the novel state-of-the-art [2]. Especially the usage of 3D endoscopes, where left and right images captured from a stereo-camera are presented on a 3D compatible monitor have proven to be beneficial, since they enable better perception of the depth, and spatial relations of structures in the scene. Additionally, exploiting the stereo information enables quantitative analyses of downstream tasks for example in 3D pose estimation.

Towards this end and building upon the challenge in the previous year [7], the AdaptOR challenge 2022 proposes a  task of novel view synthesis for endoscopic data. During training, the participants are provided the left and the right stereo camera images, and the test time task is to predict the corresponding image for a given image from the left camera. Clinically, this improves the perception of crucial structures in the surgical scene such as depth of chordae, relative position of papillary muscles, and size of mitral annulus. Furthermore, novel view synthesis is an integral sub-task of numerous existing cutting-edge depth estimation methods [5,6]. This enables not only realtime workflows, but also retrospective analyses on the existing 2D endoscopic data that would otherwise not be possible.

Intra-operative datasets in this challenge have varying camera angles, illumination, field of views and occlusions from tissues, tubes, and increase light reflections from surgical headlights. Especially demanding in these scenes, is the view-dependent appearance of the objects that are directly in front of the camera (eg. sutures). They render it difficult to train models that faithfully predict the missing image from the image pair or to define the correct correspondences between associated pixels. Therefore, the proposed task of novel view synthesis is difficult to solve.

To enhance the training split with data from a related domain, we additionally provide stereo frames captured from a mitral valve surgical simulator. This data is captured from mitral valve repair performed on patient specific 3D printed silicone valves. They contain a comparably stable illumination (less varying reflections) and stereo relation and a more standardized view angle. Participants are invited to include approaches that can learn robust image features that can be potentially transferred to the intra-operative domain (e.g., [8]).

The dataset this year is an extension of our dataset we released in the previous year, where we now consider more surgeries and additional phases of mitral valve repair to significantly increase the sizes of the single splits at higher resolutions.

References

[1] Carpentier, A., Deloche, A., Dauptain, J., Soyer, R., Blondeau, P., Piwnica, A., Dubost,C., McGoon, D.C.: A new
reconstructive operation for correction of mitral and tricuspid
insufficiency. The Journal of Thoracic and Cardiovascular Surgery 61 (1), 1–13 (1971)
[2] Casselman Filip P., Van Slycke Sam, Wellens Francis,
De Geest Raphael, Degrieck Ivan,Van Praet Frank, Vermeulen Yvette, Vanermen Hugo: Mitral Valve Surgery
Can Now Routinely Be Performed Endoscopically. Circulation 108 (10 suppl 1), II–48 (2003). DOI
https://doi.org/10.1161/01.cir.0000087391.49121.ce
[3] Maier-Hein, L., Eisenmann, M., Reinke, A. et al. Why rankings of biomedical image analysis competitions should
be interpreted with care. Nat Commun 9, 5217 (2018). https://doi.org/10.1038/s41467-018-07619-7
[4] Zhao, H., Gallo, O., Frosio, I., & Kautz, J. (2018). Loss Functions for Neural Networks for Image Processing.
ArXiv:1511.08861 [Cs]. http://arxiv.org/abs/1511.08861
[5] Watson, J., Mac Aodha, O., Turmukhambetov, D., Brostow, G. J., & Firman, M. (2020). Learning Stereo from
Single Images. ArXiv:2008.01484 [Cs]. http://arxiv.org/abs/2008.01484
[6] Hou, Y., Solin, A., & Kannala, J. (2021). Novel View Synthesis via Depth-guided Skip Connections.
ArXiv:2101.01619 [Cs]. http://arxiv.org/abs/2101.01619

[7] https://adaptor2021.github.io/ doi: 10.5281/zenodo.4646979
[8] Engelhardt S., De Simone R., Full P.M., Karck M., Wolf I. (2018) Improving Surgical Training Phantoms by
Hyperrealism: Deep Unpaired Image-to-Image Translation from Real Surgeries. In: Frangi A., Schnabel J.,
Davatzikos C., Alberola-López C., Fichtinger G. (eds) Medical Image Computing and Computer Assisted
Intervention – MICCAI 2018. MICCAI 2018. Lecture Notes in Computer Science, vol 11070. Springer, Cham, doi:
10.1007/978-3-030-00928-1_