Published May 26, 2026
| Version v1
Preprint
Open
Naive Contrastive Function Vector Injection Does Not Transfer Reasoning Capability Within the Llama-3 Family
Description
We test whether a contrastive activation difference computed at a single layer of a reasoning-distilled donor can be injected into a generally instruction-tuned host of the same architecture to transfer mathematical reasoning capability. Donor: DeepSeek-R1-Distill-Llama-8B. Host: Llama-3.1-8B-Instruct. Both share Llama-3 architecture and tokenizer, removing common cross-family confounds. We inject the contrastive vector at layer 12 with alpha in {0, 0.1, 0.3, 0.5} and evaluate on 10 held-out GSM8K problems. Baseline accuracy is 65.0%. Injection produces a monotonically decreasing accuracy: 50.0% at alpha=0.1, 30.0% at alpha=0.3, and 10.0% at alpha=0.5 (Delta = -55 percentage points relative to baseline). No cell achieves raw p<0.05 under paired exact McNemar testing; closest is p=0.0625 at alpha=0.5, which does not survive Bonferroni correction. We conclude that naive last-token contrastive subtraction does not encode a transferable reasoning capability in this single-layer setting. Reasoning-marker rate stays stable (2.00 to 2.30) across all injection cells while accuracy collapses, indicating the contrastive vector captures surface mathematical vocabulary rather than transferable computational structure. We discuss three structural hypotheses (reasoning is not a single direction; wrong layer/geometry; donor-host capability gap too small) and propose four concrete follow-up protocols. All code, calibration prompts, test prompts, per-cell generation outputs, and aggregation results are released open-source for reproduction.
Notes
Files
paper-e1-negative.pdf
Additional details
Related works
- Is derived from
- Software: 10.5281/zenodo.20352085 (DOI)
- Is part of
- Preprint: 10.5281/zenodo.20071372 (DOI)
- Is supplemented by
- Software: https://github.com/norika1207-lab/frankenstein-skeleton (URL)
Subjects
- Mechanistic Interpretability
- https://en.wikipedia.org/wiki/Mechanistic_interpretability
- Large Language Models
- Negative Results in Machine Learning