Journal article Open Access

The Case of Missed Cancers: Applying AI as a Radiologist's Safety Net

Chorev M. , Shoshan Y., Spiro A., Naor S., Hazan A., , Barros V., Weinstein I., Herzel E., Shalev V., Guindy M., and Rosen-Zvi, M.

We investigate the potential contribution of an AI system as a safety net application for radiologists in breast cancer screening. As a safety net, the AI alerts on cases suspected to be malignant which the radiologist did not recommend for a recall. We analyzed held-out data of 2,638 exams enriched with 90 missed cancers. In screening mammography settings, we show that a system alerting on 11 out of every 1,000 cases, could detect up to 10.7% of the radiologists’ missed cancers. Thus, significantly increasing radiologist’s sensitivity to 80.3%, while only slightly decreasing their specificity to 95.3%. Importantly, the safety net demonstrated a significant contribution to their performance even when radiologists utilized both mammography and ultrasound images. In those settings, it would have alerted 8.5 times per 1,000 cases, and detected 11.7% of the radiologists’ missed cancers. In an analysis of the missed cancers by an expert, we found that most of the cancers detected by the AI were visible post-hoc. Finally, we performed a reader study with five radiologists over 120 exams, 10 of which were originally missed cancers. The AI safety net was able to assist 3 out of the 5 radiologists in detecting missed cancers without raising any false alerts.

Files (386.8 kB)
Name Size
386.8 kB Download
All versions This version
Views 7474
Downloads 109109
Data volume 42.2 MB42.2 MB
Unique views 6868
Unique downloads 106106


Cite as