Inference-Subordinate Simulation: Decoupling Agent Decision Time from Playback Time in 3D Environments

This paper proposes and validates inference-subordinate simulation, an 
architectural pattern for LLM-driven agent simulation in which the 
simulation engine advances only when the agent issues a decision. 
Rather than the simulation running continuously and the agent reacting 
in real time, world state is frozen between decisions. Each decision is 
logged with full state and timing metadata. Playback reads the decision 
log and reproduces identical behavior at any speed, without re-running 
inference.

The architecture is implemented using Godot 4 as the simulation engine, 
a Python HTTP server as the inference layer, and Claude (claude-opus-4-5) 
as the vision-language agent. The agent observes a 3D environment through 
a bird's eye camera and issues structured tool calls that advance world 
state. Empirical validation across 26 runs comprising 222 decisions 
demonstrates a 100% goal success rate, a mean inference time of 4.53 
seconds per decision, and replay of the same decision sequences in under 
one second at 10x speed.

The paper documents the model selection process honestly, including the 
failure of a 7B local model on spatial reasoning tasks, as evidence that 
navigation quality is model-dependent while the architectural pattern is 
model-agnostic. Applications in cinematic production, video game NPC 
pre-computation, reinforcement learning training data generation, and 
social science modeling are discussed.

Implementation, dataset, and replay system available at: 
https://github.com/fakhtar/tilismpoc1