Published October 28, 2026 | Version v1

What Breaks in Small Generative Process Verifiers: Verdict Aggregation, Calibration Limits, and Discriminative–Generative Cascades

Authors/Creators

Description

We study a 1.5B generative process verifier (DeepSeek-R1-Distill-Qwen-1.5B with a fixed step-judgment format) on PRM800K and find that its failures lie mostly where accuracy numbers do not point. (1) Step accuracy is a base-rate mirage. A trivial all-"correct" judge scores 0.904 on the test steps; our SFT verifier scores 0.897 while flagging 15 of 1,857 steps, one correctly. Its verdicts mirror the majority class, and neither GRPO (every sampled group agrees, so advantages vanish) nor decode-time derivation repairs this. Training on multipath-consistent traces does (solution accuracy 0.417 → 0.583, McNemar p ≈ 10^-5). (2) The verifier knows more than it says. Verdict-token probabilities reach step AUROC 0.81, and noisy-OR aggregation over them gives 0.883 solution accuracy at ECE 0.092 with no fitting (SFT v2, 103 test traces), while the emitted confidence is two-valued by target design. (3) Step-label interference is real but small. Teacher-forcing an "incorrect" judgment at step k leaves later hard verdicts unchanged (CIs exclude drag increases above 0.4 points) but raises later steps' P(incorrect) by about one probability point (SFT v2: +0.9, 95% CI [+0.1, +1.9]). (4) Cascades trade cost for accuracy. Escalating only the traces a discriminative PRM is unsure about to the token-probability readout reaches 0.820 accuracy on 891 traces (the 103 test traces plus 788 assembled from their candidate pools) while calling the generative verifier on 20.5% of them. This beats the discriminative PRM alone (0.744; +7.7 points, problem-clustered 95% CI [+3.6, +12.3]) and matches running the readout on every trace within noise (0.773; +4.6 points, CI [-1.7, +11.4]) at a fifth of its generative cost.

Files

paper-6.pdf

Files (327.0 kB)

Name Size
md5:c51ed25d9c3312a2cdb30b2894a205b7
327.0 kB Preview Download

Additional details

Related works

Software

Repository URL
https://github.com/Peoplerepabulic/small-generative-process-verifiers
Programming language
Python
Development Status
Active

References

  • Khalifa, M., et al. Process Reward Models That Think. arXiv:2504.16828, 2025.
  • Lightman, H., et al. Let's Verify Step by Step. arXiv:2305.20050, 2023.
  • Wang, P., et al. Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations. ACL, 2024. Zhang, L., et al. Generative Verifiers: Reward Modeling as Next-Token Prediction. arXiv:2408.15240, 2024.