Published May 26, 2026 | Version v1

Naive Contrastive Function Vector Injection Does Not Transfer Reasoning Capability Within the Llama-3 Family

Authors/Creators

  • 1. Independent researcher, charenix.com

Description

We test whether a contrastive activation difference computed at a single layer of a reasoning-distilled donor can be injected into a generally instruction-tuned host of the same architecture to transfer mathematical reasoning capability. Donor: DeepSeek-R1-Distill-Llama-8B. Host: Llama-3.1-8B-Instruct. Both share Llama-3 architecture and tokenizer, removing common cross-family confounds. We inject the contrastive vector at layer 12 with alpha in {0, 0.1, 0.3, 0.5} and evaluate on 10 held-out GSM8K problems. Baseline accuracy is 65.0%. Injection produces a monotonically decreasing accuracy: 50.0% at alpha=0.1, 30.0% at alpha=0.3, and 10.0% at alpha=0.5 (Delta = -55 percentage points relative to baseline). No cell achieves raw p<0.05 under paired exact McNemar testing; closest is p=0.0625 at alpha=0.5, which does not survive Bonferroni correction. We conclude that naive last-token contrastive subtraction does not encode a transferable reasoning capability in this single-layer setting. Reasoning-marker rate stays stable (2.00 to 2.30) across all injection cells while accuracy collapses, indicating the contrastive vector captures surface mathematical vocabulary rather than transferable computational structure. We discuss three structural hypotheses (reasoning is not a single direction; wrong layer/geometry; donor-host capability gap too small) and propose four concrete follow-up protocols. All code, calibration prompts, test prompts, per-cell generation outputs, and aggregation results are released open-source for reproduction.

Notes

© Copyright 2026 Chen, Ho Yiing (ORCID: 0009-0006-6816-9891), Independent researcher, charenix.com. All rights reserved by the author. This work is licensed under the Creative Commons Attribution 4.0 International License.

Files

paper-e1-negative.pdf

Files (156.3 kB)

Name Size
md5:b4b530251b72c7d3846a6922b7d7f1c5
142.6 kB Preview Download
md5:c557e9b684e716cf61ef0c20795a2d05
11.8 kB Download
md5:90aeddf58cc5ac0fccd8521589957dc4
1.6 kB Preview Download
md5:90bb5318b7681c6ae9653338e81877e8
299 Bytes Preview Download

Additional details

Related works

Is derived from
Software: 10.5281/zenodo.20352085 (DOI)
Is part of
Preprint: 10.5281/zenodo.20071372 (DOI)
Is supplemented by
Software: https://github.com/norika1207-lab/frankenstein-skeleton (URL)

Subjects

Mechanistic Interpretability
https://en.wikipedia.org/wiki/Mechanistic_interpretability
Large Language Models
Negative Results in Machine Learning