Beyond Static Benchmarks: A Critical Analysis of Spatial Intelligence Evaluation in Large Language Models and the Need for Embodied Metacognitive Assessment

The recent evaluation of GPT-5's spatial intelligence by Cai et al. (2025) represents a significant attempt to assess spatial cognitive capabilities in large language models (LLMs). However, this paper argues that their framework suffers from fundamental theoretical and methodological flaws that undermine its validity. We identify three critical deficiencies: (1) the absence of grounding in established cognitive science theories, (2) the exclusion of essential spatial cognitive functions including navigation and embodied cognition, and (3) the failure to address metacognitive capabilities necessary for genuine spatial understanding.

We demonstrate that the six-component taxonomy proposed by Cai et al. lacks theoretical justification and fails to capture the dynamic, embodied nature of spatial cognition. Furthermore, we argue that static benchmarks cannot adequately assess spatial intelligence, which fundamentally requires active exploration, sensorimotor integration, and metacognitive awareness.

This paper proposes an alternative assessment paradigm based on embodied cognition and dynamic process evaluation. We introduce a five-level optical illusion protocol that reveals the metacognitive limitations of current LLMs and predict specific failure patterns at the transition from static to dynamic spatial tasks. Our analysis suggests that achieving genuine spatial intelligence in artificial systems requires not merely larger models or more data, but fundamentally different architectures that incorporate self-referential processing, uncertainty tolerance, and possibly phenomenal consciousness. The implications extend beyond technical challenges to philosophical questions about the nature of understanding and the requirements for artificial general intelligence.