Conference paper Open Access

FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos

Shawn Mathew; Saad Nadeem; Arie Kaufman

Abstract:

Haustral folds are colon wall protrusions implicated for highpolyp miss rate during optical colonoscopy procedures. If segmented ac-curately, haustral folds can allow for better estimation of missed surfaceand can also serve as valuable landmarks for registering pre-treatmentvirtual (CT) and optical colonoscopies, to guide navigation towards theanomalies found in pre-treatment scans. We present a novel generativeadversarial network, FoldIt, for feature-consistent image translation ofoptical colonoscopy videos to virtual colonoscopy renderings with haus-tral fold overlays. A new transitive loss is introduced in order to leverageground truth information between haustral fold annotations and virtualcolonoscopy renderings. We demonstrate the effectiveness of our modelon real challenging optical colonoscopy videos as well as on texturedvirtual colonoscopy videos with clinician-verified haustral fold annota-tions. All code and scripts to reproduce the experiments of this paper will be made available via our Computational Endoscopy Platform at https://github.com/nadeemlab/CEP

 

Model Weights for FoldIt

This was trained where domain A is optical colonoscopy, domain B is haustral fold annotation and domain C is virtual colonoscopy ground truth corresponding to the haustral fold annotations.

Testing Videos

The frames that were used for testing are included as well. Please cite the appropriate papers when using them.

Incetan, K., Celik, I.O., Obeid, A., Gokceler, G.I., Ozyoruk, K.B., Almalioglu,Y., Chen, R.J., Mahmood, F., Gilbert, H., Durr, N.J., et al.: VR-Caps: A virtualenvironment for capsule endoscopy. Medical Image Analysis. p. 101990 (2021)

Ma, R., Wang, R., Pizer, S., Rosenman, J., McGill, S.K., Frahm, J.M.: Real-time3D reconstruction of colonoscopic surfaces for determining missing regions. Inter-national Conference on Medical Image Computing and Computer-Assisted Inter-vention pp. 573–582 (2019)

Files (307.2 MB)
Name Size
latest_net_G_A.pth
md5:89fdaff117688717a6929756214e31ed
71.1 MB Download
latest_net_G_AC.pth
md5:b48f385700a208ac0b387c5bbbd98813
71.1 MB Download
latest_net_G_B.pth
md5:8eb96f11c002534a7d7b66d02512f691
71.1 MB Download
latest_net_G_BC.pth
md5:78d8fffd08ca088da023baba880b2cfa
71.1 MB Download
unc_vids.zip
md5:2f4414b9b1e3738acd56a0be36a57c27
11.0 MB Download
vr_caps.zip
md5:ff81c6fa58e42dbb8b7d42ac283ffda9
11.9 MB Download
95
73
views
downloads
All versions This version
Views 9595
Downloads 7373
Data volume 3.6 GB3.6 GB
Unique views 8181
Unique downloads 2828

Share

Cite as