Published August 27, 2026 | Version 1.1.0

Replication package for "A Memorization Floor for LLM Refinement of Decompiled Code"

Description

We introduce a memorization floor: a within-item control separating what LLM refinement of decompiler output recovers from its input from what it recovers from its prior. Refine   a function, then refine it again from an input whose identifiers have been destroyed, and measure what survives. Because the comparison is within-item, corpus difficulty cannot   contribute; it costs twenty API calls. Applied to functions written after our analysis plan was committed, so no released model could have memorized them, it reports two things.   Recovery is real: refined output sits +0.072 to +0.137 above an arm-matched permutation null built from its own output vocabulary. But it does not depend on the input we ablate:   destroying the input's dataflow changes the naming gain by +0.001 (95% CI [-0.026, +0.026]), and removing type prefixes or permuting names changes it by no more. A second refiner   from another vendor, registered in advance and given byte-identical inputs, reproduces this — twelve contrasts, two models, twelve nulls. Readability stays at ceiling   throughout, so a reader is given no signal. The null is bounded, not absolute: contributions under 0.056 are invisible, and the ablation leaves operations intact, so naming from   those alone remains a competing reading. No registered hypothesis was confirmed, and we report the five instrument failures behind that in full, including a reassembly harness   biased against the treated arm and an equivalence checker we registered without checking it worked on our inputs.    

This deposit is the replication package: corpus (both tiers), full pipeline, pre-registered analysis plan, the complete dated research log including every correction and   deviation, per-run configuration and raw API responses for all runs including the 270-call cross-vendor replication, and a per-number provenance map (paper/NUMBERS.md) tying   every figure in the paper to a committed producer. The manuscript itself is not included; see LICENSE-DATA.    

Changes in v1.1.0: corrects the paper's account of why the registered equivalence checker did not discriminate (a control-dependence cascade, established in correspondence with   the tool's authors) and reattributes that finding from the tool to our own procedure; adds an exploratory control-dependence sensitivity analysis; and reflects a restructure and   retitle of the manuscript. No previously reported number changed.

Files

llm-decompilation-artifact-2026-08-27.zip

Files (9.6 MB)

Name Size
md5:6487ae06ad519b30ec54f89dd0a011f5
9.6 MB Preview Download