The Information Geometry of Tiny Recursive Models Comparative Analysis with Large Language Architectures

This paper rigorously examines the pivotal shift in artificial intelligence research from the prevailing 'Scaling Hypothesis'—which governed massive Large Language Models (LLMs) with trillions of parameters—to a new paradigm centered on 'Algorithmic Depth,' exemplified by Tiny Recursive Models (TRMs) (ranging from 5 million to 30 billion parameters). LLMs, operating as purely dissipative thermodynamic systems that minimize free energy in a single forward time-step, are shown via Information Geometry to be intrinsically limited in complex, System 2-like deliberative reasoning due to their linear, autoregressive topology. In contrast, TRMs achieve superior fluid intelligence and constraint satisfaction by utilizing a single, compressed network for continuous latent contemplation and iterative refinement, employing recursive latent loops, Deep Supervision, and Adaptive Computation Time (ACT). Comparative benchmarking, particularly on the Mamun, SM (2026) …………………………………………………….…1 The Information Geometry of Tiny Recursive Models Comparative Analysis with Large Language Architectures ARC-AGI and Sudoku-Extreme tasks, demonstrates that 7-million parameter TRM architectures dominate frontier LLMs, achieving up to a 300-fold reduction in energy consumption per task.

Also available on Academia.edu: https://www.academia.edu/attachments/132367869/download_file?s=portfolio