From Topology to LLM Agent Skills: Vector-Quantized Geodesic Trajectories as Skills for Spatial Aware LLM-Driven Agents
Description
LLM-based agents are often criticized for lacking spatial understanding and mainly exploiting statistical text patterns. We investigate their spatial comprehension through an architecture combining geometrical tools with a large language model serving as a high-level orchestrator in grid-world environments. The agent first collects geodesic trajectories, which are then vector‑quantized to extract a representative subset. Each resulting trajectory defines a reusable tool. The LLM is used offline to interpret the learned skills by projecting them into natural language descriptions of the underlying behavioral patterns. Online, the LLM chooses the appropriate tool conditioned on the current state and goal. Low-level control is handled by primitive actions that execute the selected skill. From an agentic AI perspective, this approach separates learning into two levels. Skill discovery is handled through unsupervised quantization of trajectories, while reasoning and decision-making are handled by the LLM.