An O(1) Modular Fingerprint of Input Norm Predicts Spectral Concentration in Transformer Hidden States — and the Correlation Sign Flips with Rotary Embeddings

We introduce an O(1)-time, dimension-independent diagnostic computed solely from raw token IDs that predicts spectral concentration in transformer final-layer hidden states. The diagnostic exhibits a reproducible sign flip from positive correlation under absolute positional embeddings to negative correlation under rotary embeddings (RoPE), with the negative correlation strengthening with model scale. Partial regression controlling for sequence length shows the signal is an independent geometric feature in large (>2B) RoPE models, while acting as a high-speed length proxy in smaller ones. A proof-of-concept routing gate using the diagnostic yields 57.2% fewer tokens processed and 53.0% lower energy consumption on Apple Silicon. Isolated tests confirm the inference bottleneck is memory-bound. 

# Modular LLM Diagnostic Reproducibility Package Paper: An O(1) Modular Fingerprint... (Lynch, 2025)

## Setup pip install torch transformers scipy numpy pandas statsmodels tqdm

## Reproduce Key Results - env_test.py: Verify Torch/MPS. - ROPEmodelTest.py: TinyLlama correlation (r=-0.287). - ROPEmodelTest2.py: Mistral-7B (r=-0.420). - test_models.py: GPT2/Phi-2. - extractor.py: Extract CSVs from logs. - CSV.py: Generate Table 1 + confounders. - measure_mac_01.py: Gated routing (57% savings; sudo needed for powermetrics). - measure_mac_isolated_test.py: Latency/energy tests. - proof3.py: Standalone λ₂ calc.

prompts.txt/simple_prompts.txt: Test datasets.

Raw logs/JSONs: Gating outputs.