Structured Reasoning in LLM Optimization Agents: Scaffolding, Not Regularization
Description
LLM-based optimization agents increasingly produce structured reasoning artifacts — hypothesis summaries, causal models, prediction logs — that persist across iterations. The assumption is that forcing articulation regularizes reasoning, as the self-explanation effect suggests it does for human learners. We test this assumption using SynthOracle, a family of synthetic multi-objective optimization oracles with known causal structure that enables separate measurement of optimization quality and reasoning quality. We pair the benchmark with a verbal regularization (VR) protocol that requires the agent to hypothesize, predict, and reconcile at each iteration via a structured summary embedded in conversation context. Through controlled ablations on two oracles and two models (Claude Opus 4.6 and Sonnet 4.6; n = 5 paired seeds per condition), we find that the forced summary is scaffolding, not regularization: it improves optimization for a less capable model (Sonnet, +26%) and a harder task (12-dimensional with noise, +14% to +35%), but hurts the most capable model on the simplest oracle (Opus, -35%, p = 0.0006). The pattern is consistent across 18 of 20 paired seeds (binomial p = 0.0002). A mechanism-isolation experiment shows that the performance penalty comes from the summary's persistence in conversation context, not from the cognitive cost of producing it: an agent that produces the summary but does not receive it back performs identically to one that never produces it (p = 0.35). We connect this finding to the architectural trend in frontier reasoning models toward ephemeral chain-of-thought traces and derive a design principle: match the rigidity of your reasoning protocol to the gap between model capability and task difficulty.
Files
synthoracle_v1.pdf
Files
(443.2 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:a5e768cf5403a89c55d74f5cd6737462
|
443.2 kB | Preview Download |
Additional details
Dates
- Submitted
-
2026-04-20Preprint uploaded
Software
- Repository URL
- https://github.com/kar-ganap/synthoracle
- Programming language
- Python
- Development Status
- Active