Published May 21, 2026
| Version v0.7.2
Software
Open
Predictable Benchmarks? An Evaluation-Protocol-Adjusted Residual Audit of Public LLM Benchmark Scores
Authors/Creators
Description
Publication-polish release addressing reviewer-style feedback.
- Added paired bootstrap confidence intervals for Full and Scale-only OOF MAE and their improvement.
- Renamed leave-one-benchmark metrics in
summary.jsonto distinguish pooled held-out-cell MAE from the unweighted mean of per-benchmark fold MAEs. - Added ordinal recipe-code perturbation sensitivity for
math_intensity,code_pretrain, andsynthetic_quality. - Added
data/model_feature_coding.csv/outputs/model_feature_coding.csvand a feature-coding rubric. - Added explicit sparse-benchmark disclosure near the benchmark residual table.
- Added AI-use disclosure to the paper and a separate disclosure document.
- Clarified that closed-source Gemini scale values are anchoring proxies, not parameter estimates.
- Cleaned repository hygiene: single README, canonical
paper/main.tex,LICENSE, filledCITATION.cff, and removed deadbenchmark_suite_v3.py.
This release exists to mint the Zenodo concept DOI. The next release (v0.7.3) embeds that DOI into the paper.
Notes
Files
edherranz/llm-benchmark-residual-audit-v0.7.2.zip
Additional details
Related works
- Is supplement to
- Software: https://github.com/edherranz/llm-benchmark-residual-audit/tree/v0.7.2 (URL)
Software
- Repository URL
- https://github.com/edherranz/llm-benchmark-residual-audit