Human Before the Loop
Description
Multipurpose AI agents come with many security risks, including prompt injection, which can instruct a large language model (LLM) to find and deliver sensitive data to an attacker. Current proposed defenses carry their own trade-offs: security middleware or post-training to refuse malicious instructions can fail to predict novel attacks, runtime supervision slows efficiency, and restructuring control flow has been demonstrated to lower task completion. To address this, we present Human Before the Loop, an agent framework that determines the action space and data access in the design phase, leaving no runtime path open to sensitive data or destructive actions. Rather than relying on traditional open-ended tool calling, our framework navigates actions within nested menus using a Hierarchical Extended Finite State Machine (HEFSM). Final menu choices are bound to prewritten deterministic Python scripts that execute the intended action, we term this simplification semantic action abstraction. This method eliminates the risk of data exfiltration while increasing reliability and realized capability. In our security evals our framework prevented all prompt injection attacks without a decrease in benign task completion, whereas the baseline tool-calling agent failed to prevent 8.5% of attacks and fell to 86.6% task completion. A least privilege tool calling agent also prevented all attacks but saw task completion fall to 66.2%. In a reliability eval, general task completion saw 76.5% with the tool-calling agent and rose to 81.4% with our framework. To demonstrate capability we tasked agents with completing the Pokémon Red tutorial phase, our framework enabled a 30B parameter model to complete the phase in 341 turns, while the same model using a tool-calling harness made no meaningful progress across 1,000 model calls.
Files
HumanBeforeTheLoop_Preprint.pdf
Additional details
Software
- Repository URL
- https://github.com/ella0333/human-before-the-loop
- Programming language
- Python
- Development Status
- Active