SACRO: Semi Automated Checking of Research Outputs: Technical Deliverables

We report on the software deliverables produced by the DARE UK Driver project Semi Automated Checking of Research Outputs (SACRO). This project brought together Trusted Research Environments (TREs) with software developers and Machine Learning researchers and practitioners to produce a suite of tools to address (i) a current bottleneck in checking research outputs for privacy leakage, and (ii) an impending issue that TREs do not have the resources or skills to risk-assess trained machine learning models.

The tools produced broadly split into three parts. The first is a library of 'drop-in' like-for-like replacements for researchers, that replace commonly used query commands in Python/R and Stata with acro versions that conduct and report on disclosure risk assessments at the same time as reporting the results of the original queries.

The second is a viewer for TRE output checkers to use that ingests the requested outputs alongside the description of their disclosure risk to facilitate the checking and release process.

The third component is a range of tools for (i) encouraging researchers to consider disclosure risk throughout their Machine Learning workflow, (ii) running a range of different types of 'attacks' on trained models and (iii) co-ordinate running attacks on a model a researcher has requested to egress, and produce a report and recommendation for TRE staff.