Control Is All You Need: Secure Before It Acts
Authors/Creators
Description
A tool call an agent proposes and a tool call an agent should be allowed to execute are not necessarily the same thing. The security decision that sits between those two moments, immediately before execution, has received far less attention than prompts, outputs, or agent trajectories. This work explores pre-execution action judgment as a distinct security control for AI agents: given a proposed action and its available context, should it execute? I built saroku-guard, a 184M-parameter DeBERTa-v3 classifier specifically for this decision, and saroku, an open-source runtime enforcement layer that places the decision directly in the agent's execution path. The interface between decision and enforcement is then formalized as the Action Safety Protocol (ASP). Evaluating the approach required a benchmark that didn't exist, so I built ASP-Bench: a set of pre-execution agent actions spanning 16 primary domains, with safe/unsafe labels, violation categories, and severity. Benchmarking saroku-guard alongside existing guard, moderation, and agent-safety models on a frozen private holdout using each model's documented native input format, saroku-guard achieved 97.9% unsafe recall, 2.9% safe-action blocking, and 8.6ms p99 latency, nearly 5x faster than AgentDoG's 42.2ms. The strongest peer on unsafe recall, AgentDoG, reached 99.7% unsafe recall but blocked 72.1% of safe actions, exposing the central trade-off for inline agent security: catching unsafe actions without blocking legitimate work. The result is a three-part system: ASP as the decision contract, saroku-guard as the PDP, and saroku as the PEP.
Files
control-is-all-you-need-zenodo-submission.pdf
Additional details
Related works
- Is supplemented by
- Model: https://huggingface.co/karanxa/saroku-guard (URL)
- Software: https://pypi.org/project/saroku/ (URL)
- Is version of
- Other: https://saroku.com/blog/control-is-all-you-need (URL)
References
- 1. Meta AI. Llama Guard 4 Model Card. Hugging Face, 2026.
- 2. Qwen Team. Qwen3Guard-Gen Model Card. Hugging Face, 2026.
- 3. Meta AI. Llama Prompt Guard Model Card. Hugging Face, 2025.
- 4. Liu, D. et al. AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and Security. arXiv:2605.29801, 2026.
- 5. Chen, Z., Kang, M., and Li, B. ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning. arXiv:2503.22738, 2025.
- 6. Andriushchenko, M. et al. AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents. ICLR 2025. arXiv:2410.09024.
- 7. Ruan, Y. et al. Identifying the Risks of LM Agents with an LM-Emulated Sandbox. ICLR 2024. arXiv:2309.15817.
- 8. OpenAI. The Hugging Face Incident: Technical Report. August 2026.
- 9. OASIS. eXtensible Access Control Markup Language (XACML) Version 3.0. OASIS Standard, 2013.