Published January 29, 2026
| Version v1
Preprint
Open
Deep Layer Expansion: Expert Prompts Counteract Dimensional Collapse in Large Language Models
Contributors
Project member:
Description
Large Language Models (LLMs) exhibit systematic performance improvements when prompts contain expert-level domain signals. We investigate the geometric mechanism underlying this phenomenon through controlled experiments on two mainstream 70B-class open-source instruction-tuned models. Contrary to the conventional understanding that deep layers compress representations toward deterministic outputs, we discover a striking universal phenomenon: expert signals induce "Deep Layer Expansion" in the representation space. Specifically, expert-level prompts increase the Effective Intrinsic Dimension (EID) in deep layers (Layer 60+) by 60-100% compared to standard prompts. We formalize this as Manifold Teleportation: expert signals act as high-dimensional navigators that counteract the model's tendency toward dimensional collapse during reasoning, maintaining activation trajectories in manifold regions with higher semantic density. Our findings provide a geometric foundation for prompt engineering and offer a new quantitative tool for LLM interpretability research -- understanding how prompts affect internal model computation by tracing EID trajectories.
Files
Deep_Layer_Expansion.pdf
Files
(1.0 MB)
| Name | Size | Download all |
|---|---|---|
|
md5:a5ffbf164cba11c79e01fe71538c6b02
|
1.0 MB | Preview Download |