Published March 2, 2026 | Version v1
Preprint Open

Shaped Noise Injection at Inference Time: Domain Precision, Loop Breaking, and the Terminal Measurement Limit

Authors/Creators

  • 1. Cage & Mirror Press

Description

Can noise shaped to domain-discriminative directions in activation space control inference-time output distributions? This paper tests the thesis across ten experimental phases on Qwen 2.5 models from 0.5B to 7B parameters, using INLP-discovered domain directions as the noise shaping basis. Shaped noise achieves modest domain-specific entropy reductions (up to 6.1% for legal at 7B) and breaks 100% of repetition loops at both 3B and 7B, outperforming temperature scaling and matching repetition penalty on escape rate while achieving near-perfect token uniqueness (0.99+). However, cross-domain selectivity is fundamentally limited. All correction attempts — scalar cancellation, subspace decomposition, and optimal linear correction via matrix inversion — fail. The root cause is identified as the terminal measurement problem: the response matrix characterizes the system's input-output mapping but cannot invert the nonlinear transformations that generate cross-domain bleed during the forward pass. In d=3,584 dimensions, concentration of measure guarantees that geometric orthogonality of INLP directions is uninformative about functional overlap. This result constrains the entire class of direction-space intervention methods.

Notes

Paper 24 in the Structural Compression Theory research program. Series: Activation Geometry.

Files

shaped_noise_injection.pdf

Files (371.2 kB)

Name Size Download all
md5:96adb34c5f5284a7df4d5ed954cd4f43
371.2 kB Preview Download