Pure W-Expansion / NecroGraft: Full-Rank Weight Growth for Zero-Forgetting Model Adaptation

I present Pure W-Expansion (NecroGraft), a weight-adaptation method that augments frozen pretrained models with full-rank trainable expansion matrices, enabling domain specific learning without catastrophic forgetting. Unlike Low-Rank Adaptation (LoRA), which approximates weight updates in a compressed subspace, W-Expansion grows the weight matrix in a new full-rank dimension, preserving the original weights exactly. After training, expansion weights are concatenated with frozen base weights into a single matrix, yielding zero inference overhead. A learnable Yaka Gate (λ) with RMS input scaling stabilises mixed-precision training; both are absorbed during merging and disappear at inference. I formalise the combine_expansions operation and prove the Zero-Interference Theorem, which guarantees that multiple independently trained domain expansions can be merged into one weight matrix without cross-domain contamination.

Empirically, on SmolLM2-1.7B-Instruct (high-school biology adaptation), NecroGraft (g=128, all 6 modules, all 24 layers) improves MMLU retention from 0.3525 to 0.4335 (+8.1 pts), surpassing LoRA (r=64) which reaches 0.4009 (+4.8 pts). Domain perplexity drops from 6.03 to 4.83, matching LoRA's 4.79. On DeepSeek-Coder-1.3B-Instruct (medical QA adaptation), NecroGraft drives MedQA perplexity from 20.68 to 4.60 (2-epochs) versus LoRA's 5.53; even with 5 epochs the graft achieves 4.60 while LoRA saturates. MMLU-STEM retention improves with training (0.0857 → 0.1170 over two epochs), and the Yaka Gate learns to amplify grafts in mid-depth layers (>1.0) while clamping down in the final layer (0.50–0.77 at Layer 23) to form an automated firewall. A layer-wise autopsy reveals that residual interference from the graft, accumulated in hidden states before reaching the output gates, is the root cause of remaining output interference on extreme domain shifts, a phenomenon distinct from catastrophic forgetting, which is structurally impossible by construction. The Slice variant achieves true zero-overhead inference by geometric expansion; the Learned variant, when deployed as a static additive merge, also yields base level latency (0.0457 s). My NecroGraft the Pure W-Expansion is not merely a fine-tuning method: it is a new paradigm I proposed, tested and validated myself and it proves a structural difference for how models should acquire knowledge not by being rewritten, but by growing.

Code & Installation:
GitHub Repository: https://github.com/ElBalor/Necrograft
PyPI Package: pip install necrograft