PrivacyPR: A Dataset of Real-World Privacy Fixes and Their Review Discussions in GitHub Pull Requests
Description
PrivacyPR contains 4,131 merged GitHub pull requests (June 2018 to January 2024) from 1,469 repositories, each annotated by an LLM (Gemini 2.5 Flash-Lite) for whether it fixes a privacy issue, its fix types, and the GDPR Art. 5(1) principles it serves; only the privacy label of the Verified split is human-certified. It has three splits: Full (2,819 PRs: the 1,169 fixes labeled with the highest confidence, 1,643 non-fix PRs and 7 unlabeled ones; the 1,312 lower-confidence fixes are released separately), Verified (250 privacy fixes and 250 non-fix PRs whose privacy label two human annotators accepted) and Small (the first 100 per class of Verified).
Patch text is replaced by SHA-256 digests and can be re-fetched from GitHub and verified with the included script; contributor logins and names are removed, and e-mail addresses and the names written next to them are replaced. privacypr_data.zip holds the data (JSON Lines) and the annotators' verdicts; privacypr_code.zip holds the collection pipeline and the evaluation harness with all prompts; privacypr_results.zip holds the reviews, judge verdicts and analysis of the paper's example evaluation on Small. See README.md.