Designing for Structural AI Failure: Interface Stress Testing as a Diagnostic Instrument for Physical Grounding

Current multimodal AI systems exhibit reproducible, diagnosable architectural class failures in spatial reasoning, physical coherence, and temporal consistency. These represent hallmarks of current transformer architectures rather than model-specific artifacts. These failures are collectively described as the Inversion Error [1], a diagnosis independently supported by Ma et al. [2], who report that current 3D large language models systematically fail to understand 3D spatial relationships. The failures are invisible from within the systems that produce them not only because the architecture lacks the physical grounding layer (Enactive floor) that detection would require, but because that absence is structural rather than incidental. They are visible from outside them because the embodied human observer stands on exactly that layer. This position paper proposes to exploit this diagnostic asymmetry by reorienting AI-focused Human-Computer Interaction design practice: from designing interfaces that smooth over AI failure modes, to interfaces that deliberately expose them.

The Inversion Error position paper [1] established the first designer role in this two-paper socio-technical systems research program: the socio-technical systems designer as diagnostic architect, exposing the structural condition in current transformer architectures from outside the engineering system. That paper diagnosed and named the condition; this paper proposes the intervention at the level of the second designer role. Where the diagnostic architect identifies what is wrong with the architecture, the interface designer as More Knowledgeable Other (MKO) [4] provides what it structurally lacks: the embodied physical grounding that makes AI outputs coherent in a physical world. The mechanism for the embodied physical grounding is Reinforcement Learning from Physical Feedback (RLPF) [1]: encoding physical constraints as differentiable priors inside the neural architecture rather than imposing them as post-hoc corrections on its outputs.

Building on the HCI tradition, within which screen-based interaction remains the dominant paradigm, I propose a two-phase stress testing program. Phase 1, the Spaghetti Table Protocol Challenge, operates at the screen-based level: a distributed research effort that scales diagnostic methodology across the design and HCI research communities using existing interfaces. Phase 2, the Universe Falling Apart Protocol, extends this program into embodied XR environments, grounded in the emerging premise of Human+Computer (H+C) Immersion [3]: that the human user and the computational system can be co-present in a single environment where the symbolic and the embodied are united rather than separated by an interface layer. Where screen-based interaction exposes AI grounding failures symbolically, embodied interaction exposes them physically, generating the RLPF signal at the level of direct sensorimotor engagement. Shneiderman's principle that governance must match autonomy level [5] provides the regulatory frame; H+C Immersion provides the ontological one. 2D screen-based HCI interface stress testing offers an excellent starting point for the methodology; 3D multimodal immersion is the next step.

Drawing on the Chaos Engineering tradition in software reliability [6], I introduce the Chaos Monkey for the Inversion Error: a class of interface designs that leverage human embodied cognition as a diagnostic instrument for detecting AI architectural failures. I present the Spaghetti Table Protocol as a proof-of-concept screen-based stress testing methodology, reporting an aggregate diagnostic score of 4 out of 30 across three leading multimodal systems. The logged outputs of this stress testing constitute training data for RLPF [1]: the mechanism through which the interface designer as MKO provides the physical grounding signal that current AI training pipelines do not collect. Phase 2, the Universe Falling Apart Protocol, is inspired by Philip K. Dick's argument about the ontological persistence of reality turned into a diagnostic criterion: an AI-generated environment passes the stress test if it holds together under embodied, multimodal human interaction.[1] Together, the two phases produce the dataset and the RLPF training signal that foundation model architectures require to address the Inversion Error at the level of structure rather than output.