Ruler Score Discrepancies in Llama-3.1-8B Benchmark Evaluations Across Studies

This report synthesises findings from 1 peer-reviewed paper addressing the following research question: Benchmark archaeology: investigate Ruler score discrepancy for Llama-3.1-8B --- reported 1.9\%--85.6\% (spread 83.7pp) across 2 papers. Sources: 'MTraining: Distributed Dynamic Sparse At' (1.9\%);. 8 claims were extracted from source literature; 8 were independently verified against retrieved documents. An automated multi-reviewer quality assessment produced a score of 9.0/10. This report is a machine-generated literature synthesis and does not constitute original research.

Research goal: Benchmark archaeology: investigate Ruler score discrepancy for Llama-3.1-8B — reported 1.9%–85.6% (spread 83.7pp) across 2 papers. Sources: 'MTraining: Distributed Dynamic Sparse At' (1.9%); 'ReST-KV: Robust KV Cache Eviction with L' (85.6%). Identify evaluation protocol differences (few-shot, prompting, preprocessing).

Autonomous literature synthesis. Automated review score: 9.0/10. Full text and citation available at Assignee Research.