Hyperbolic Helices Reveal Why Transformers Can't Count: Geometric Patterns of Semantic Uncertainty

This work is NOT peer reviewed. Feedback is welcome, please open a github issue.

We discover why Large Language Models fail catastrophically at simple counting tasks - they must navigate helical-like trajectories through hyperbolic embedding space, requiring up to ~10,000× longer paths than normal text processing. Through analysis of 50,000 semantic triples across 12 datasets, we prove that transformer embeddings exhibit hyperbolic geometry (measured curvature κ ≈ -0.73, with 100% reverse triangle inequality violations in our sample).

Our key finding: counting forces models to follow constrained trajectories that manifest as helical patterns in embedding space, with path deviation following D = 2πN sinh(r_min), where N is the count and r_min is the minimum context distance. We derive this formula through variational analysis and validate it empirically using trajectory measurements from real models.

We introduce a geometric framework for understanding semantic uncertainty - distinct from output confidence - that captures when models struggle with navigation rather than knowledge. Different uncertainty types create characteristic geometric signatures: counting generates helical trajectories, complex reasoning shows high path roughness, and conceptual bridges create discontinuous jumps. Our trajectory-based detection achieves 76.9% F1 score and captures patterns completely orthogonal to traditional confidence methods (0% overlap).

This work establishes hyperbolic geometry as the root cause of iteration failures in transformers and provides both theoretical understanding and practical tools for detecting semantic uncertainty. Includes complete mathematical derivations, experimental validation code, and visualization scripts.