Code & supporting documents for "Nationwide real-world implementation of AI for cancer detection in population-based mammography screening (PRAIM)"

The PRAIM study (PRospective multicenter observational study of an integrated Artificial Intelligence system with live Monitoring) was a study conducted within the German breast cancer screening program from July 2021 to February 2023 to assess the impact of an AI-based decision support software. This Zenodo record contains the statistical analysis code for PRAIM. Please find the corresponding data in Dryad.

Context

The PRAIM study has been published in Nature Medicine: Nationwide real-world implementation of AI for cancer detection in population-based mammography screening. Please refer to the article for further information on study design, results, and discussion of impact. The study has been previously registered in the German Clinical Trials Register and the study protocol can be found on the website of the University of Luebeck.
Below you can find information contextualizing the study and how it was embedded within the German breast screening program.

Context: Mammography screening in Germany

In Germany, women aged 50-69 are invited to breast cancer screening every two years. Each round of screening is called a screening examination. The goal of screening is to detect breast cancer early while it's still easily treatable and subsequently improve outcomes for women. After the mammography (x-ray images of the breasts), two radiologists independently assess the images (called a read) – with additional prior images from earlier screening rounds if available – to look for suspicious tissue in the breast. Optionally, there can be a third supervising radiologist, especially if one of the first two radiologists is still inexperienced. Each radiologist independently decides whether the examination is suspicious (warranting further diagnostic tests) or unsuspicious (screening process stops for the woman here, to be restarted two years later).
If at least one of the initial radiologists deems the examination suspicious, a consensus conference is initiated. In this conference, a group of radiologists together decides whether the woman should be recalled for further investigations like e.g. mammography magnifications, ultrasound, or MRI. If the examination is still suspicious after recall, a minimally invasive pre-operation biopsy is typically initiated for histopathological confirmation. If necessary, treatment is started subsequently.

Context: Vara

Vara is a Berlin-based company which offers an AI-supported viewer to aid radiologists when assessing breast cancer screening mammographies.
When using Vara, radiologists were supported by two AI-based features:

1. Normal triaging: The software selects a subset of all mammograms (~60% of all examinations in PRAIM) that are deemed highly unsuspicious by the AI model. These examinations are tagged “normal“ in the worklist.
2. Safety net: The software selects a subset of all examinations that are deemed highly suspicious by the AI model (~1.5% of all examinations in PRAIM). Radiologists first assess the screening examination without any AI tags. Only when radiologists interpret an examination as unsuspicious, the safety net is activated with an alert and a suggested localization of the suspicious region(s) in the images. Radiologists are then asked to reconsider their decision and can either accept or reject the safety net’s suggestion.

For some screening examinations neither the normal triaging nor the safety net is active. These examinations are left unclassified and therefore are read by radiologists without AI-support. Please refer to the supplementary material of the linked article for example images.

Context: PRAIM

The data for the PRAIM study was collected from July 1st, 2021 to February 23rd, 2023 from 12 screening units in Germany. Please note that the data is observational, i.e. there was no randomization: When assessing an examination, radiologists could decide whether they want to diagnose in Vara (with AI support) or in other software without AI support. The main analysis model controls for the identified confounders via propensity scores, overlap weighting and a simple regression model. For more details, please refer to the article.

The study was approved by the ethics committee of the University of Lübeck (22-043).