[contract] rows = 3000; task_id integer-valued: TRUE; range 11-510 (500 tasks)
[contract] duplicate keys (task_id, model, prompt_strategy): 0; missing grid cells: 0
[contract] metric cells: blank = 4344, literal 'NA'/'null' = 0; failing rows with any metric = 0; failing rows with metric 0 = 0; passing rows with a missing metric = 0
[consistency] vs source_results/*.csv: unmatched = 0, pass differs = 0, metric cells differ = 0
[consistency] vs paper dataset generations.csv (R pipeline input): unmatched = 0, pass differs = 0, metric cells differ = 0
[consistency] temperature in source rows: 0.2
[repro] pass rows = 1914; tasks = 383; claude-sonnet-4-6 = 708, gemini-2.5-pro = 604, gpt-5.4 = 602
[repro] FF1 blocks = 534; Friedman chi2 = 84.48, p = 4.51e-19; Kendall W = 0.0791; identical across all three = 46.8%; all zero = 37.5%
[repro] wilcox.test(exact = FALSE, correct = TRUE): n and V reproduced in 6/6; max |dZ| = 0.00275; max relative |dp| = 0.00853; Holm p (FF1, FF3) = 4.59e-07 / 1.47e-16 / 4.59e-07 / 0.0102 / 0.0102 / 0.000855
[repro] wilcox.test(exact = FALSE, correct = FALSE): n and V reproduced in 6/6; max |dZ| = 4.44e-15; max relative |dp| = 9.58e-16; Holm p (FF1, FF3) = 4.57e-07 / 1.46e-16 / 4.57e-07 / 0.0101 / 0.0101 / 0.000849
[repro] reported Holm p = 4.57e-07 / 1.46e-16 / 4.57e-07 / 0.0101 / 0.0101 / 0.000849
[repro] FF2: n/clusters reproduced in 12/12; max |d rho| = 7.77e-16; max |d cluster CI bound| = 6.66e-16 (cluster bootstrap: sorted task_id, sample with replacement, stack rows, quantile type 7, set.seed per row)
[repro] FF2 row bootstrap (sensitivity): max |d CI bound| = 6.66e-16 (provider files in alphabetical order, original row order, sample.int, quantile type 7)
[C2] all FF1 blocks: 534
[C2] FF1 blocks inside the 224-task overlap: 285
[C2]   ... complete under pass_evalplus_plus: 224
[C2]   ... excluding the 28 tasks with an original/base discrepancy: 263
[C2]   ... excluding them and complete under pass_evalplus_plus: 224
[C2]   ... restricted to the 180 tasks whose EvalPlus base test equals the original: 246
[C2]   ... restricted to them and complete under pass_evalplus_plus: 211
