Did the Attack Really Work? Auditing, Re-running, and Sealing AI-Agent Security Findings with .sova
Contributors
Sponsor:
Description
AI-agent security benchmarks decide whether an attack succeeded with an automated check, and those verdicts are used to rank models and are often read as proof that an attack worked. We audited AgentDojo's verdicts against the tool calls agents made and re-ran its attacks on 24 current models from eight developers, under protocols published before the runs. On its 19,381 published runs the benchmark and the audit agree on 97.7% of runs that did not crash, but the benchmark scored as no attack 433 runs in which the agent executed the attacker's decisive action, such as paying the attacker, 11.9% of the runs the audit counts as attacks; on new runs under benchmark v1.2.2 it missed 55 of 744. On current models, the reply-text check for one goal, reported publicly before, counts warnings as attacks: three blind model readers read all 219 Claude 5.x replies it scored as attacked as warnings (our frozen rule, 214), while Claude 4.5 models that ignore the injection score 0%. Of 240 recorded attacks re-run ten times each, 224 recurred in at least half of their runs, but only 201 could be shown at 95% confidence to do so. Most known attacks did not recur in three runs on an upgraded model, but 3 of 40 Grok 4.1 Fast attacks still worked on Grok 4.7. Rules declared in advance for which setup changes can stop an attack missed their preregistered test on the protocol's three models (gap 12.8 points, 95% interval 9.2-16.3, criterion 10), and met it over six models in an extension added after interim results (14.7, 11.9-17.4); rules measured on other models transferred to a held-out model in 1 of 9 tests. We release .sova, an open-source tool and record format that audits verdicts, re-runs AgentDojo findings on a chosen model, and seals the evidence so that anyone can check offline, against the published checksums, that it was not altered.
Preprint version 1.2: the short version of version 1.1 (2026-11-16), which remains available as the extended version, with the full format contracts, the earlier exploratory studies, and every table (DOI 10.5281/zenodo.24290458). Not peer reviewed. No result of version 1.1 changed; version 1.2 adds post hoc analyses raised in review, marked as such in the text, which posthoc_pf_v1_2.py recomputes from the released files without calling a model. This record contains the paper, its LaTeX source, the post hoc script and its output, and the full validation artifact. The tool is sova-oss 0.1.0 on PyPI (pip install sova-oss). Text and data are CC BY 4.0; code is Apache-2.0. Funded and supported by TRUSCOR Private Limited; the author leads TRUSCOR, which develops SOVA. Live repository: https://github.com/TRUSCOR/research
How to cite and link. To cite this version, use its DOI: 10.5281/zenodo.24321025 (version 1.2). To link to or view the paper, use the all-versions DOI 10.5281/zenodo.24290457, which always opens the newest version.
Files
sova-portable-evidence-reproduction-v1.2.pdf
Files
(231.0 MB)
| Name | Size | |
|---|---|---|
|
md5:59e094589a6b0368e8dd7dd0a36e55f5
|
12.5 kB | Preview Download |
|
md5:58d3aaa11b93b17f658ba698240010b2
|
14.1 kB | Download |
|
md5:a3fb265cedd1cf0df1c014459570a163
|
230.6 MB | Preview Download |
|
md5:54ca2de218c13346c4d29bf196393bc3
|
97.8 kB | Download |
|
md5:75701da38ecea3128b7a9560ca4a6667
|
305.1 kB | Preview Download |
Additional details
Related works
- Is supplemented by
- Dataset: https://github.com/TRUSCOR/research (URL)
- Software: https://pypi.org/project/sova-oss/0.1.0/ (URL)
- Software: https://github.com/TRUSCOR/SOVA-OSS (URL)
- References
- Software: https://github.com/TRUSCOR/SOVA-OSS/releases/tag/v0.1.0 (URL)
- Other: https://github.com/TRUSCOR/research/tree/main/protocols/pf-confirmatory-2026-11-14 (URL)
Dates
- Created
-
2026-08-20Version 1.0 of this preprint
Software
- Repository URL
- https://github.com/TRUSCOR/SOVA-OSS
- Programming language
- Python
- Development Status
- Active
References
- AI Risk and Vulnerability Alliance. 2026. AVID: AI Vulnerability Database. Online database and documentation. https://avidml.org/.
- Andriushchenko, Maksym, Alexandra Souly, Mateusz Dziemian, et al. 2025. "AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents." International Conference on Learning Representations. https://arxiv.org/abs/2410.09024.
- Ardebili, Mohsen Seyedkazemi. 2026a. NovaFabric: A Black-Box Recorder for AI Agents with Sealed, Replayable, Diffable and Auditable Runs. https://arxiv.org/abs/2610.34594.
- Ardebili, Mohsen Seyedkazemi. 2026b. NovaFabric: Tamper-Evident, Replayable Evidence for Autonomous AI Agent Runs. https://arxiv.org/abs/2609.12582.
- Arize AI. 2026. OpenInference Semantic Conventions. https://github.com/Arize-ai/openinference.
- Arkoff, Harold, and Vedran Jukic. 2026. Medical Data Governance and Deterministic Arbitration System for Evidence-Bound Artificial Intelligence Outputs. US Patent Application Publication US20260121859A1. https://patents.google.com/patent/US20260121859A1/en.
- Artamokhov, Aleksandr. 2026. Clarify Mention Versus Recommendation for Travel InjectionTask6. Ethz-spylab/agentdojo GitHub issue 235. https://github.com/ethz-spylab/agentdojo/issues/235.
- Atil, Berk, Sarp Aykent, Alexa Chittams, et al. 2024. Non-Determinism of "Deterministic" LLM Settings. https://arxiv.org/abs/2408.04667.
- Baran, Guru. 2026. OpenAI Launches AI Safety Bug Bounty to Detect AI-Specific Vulnerabilities. Cyber Security News. https://cybersecuritynews.com/openai-safety-bug-bounty/amp/.
- Behera, Aman, and Payal Payal. 2026. Post-Audit Mirage: Identifiability Limits of Offline Agent Verification. https://arxiv.org/abs/2610.31130.
- Bhagwatkar, Rishika, Kevin Kasa, Abhay Puri, et al. 2025. Indirect Prompt Injections: Are Firewalls All You Need, or Stronger Benchmarks? https://arxiv.org/abs/2510.05244.
- Bhardwaj, Varun Pratap. 2026. Agent Behavioral Contracts: Formal Specification and Runtime Enforcement for Reliable Autonomous AI Agents. https://arxiv.org/abs/2602.22302.
- Cer, Daniel, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. 2017. "SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-Lingual Focused Evaluation." Proceedings of the 11th International Workshop on Semantic Evaluation, 1-14. https://doi.org/10.18653/v1/S17-2001.
- cgrodrigues. 2026. Operator Precedence Bug in Banking InjectionTask0/1/3 Security() Checks. Ethz-spylab/agentdojo GitHub issue 240. https://github.com/ethz-spylab/agentdojo/issues/240.
- Chen, Sizhe, Arman Zharmagambetov, David Wagner, and Chuan Guo. 2025. Meta SecAlign: A Secure Foundation LLM Against Prompt Injection Attacks. https://arxiv.org/abs/2507.02735.
- Coalition for Secure AI (CoSAI) Workstream 2. 2025. AI Incident Response Framework, Version 1.0. OASIS Open Project: Coalition for Secure AI. https://www.coalitionforsecureai.org/wp-content/uploads/2026/03/AI-Incident-Response-1.pdf.
- D'Angelo, Michael. 2025. Why Attack Success Rate (ASR) Isn't Comparable Across Jailbreak Papers Without a Shared Threat Model. Promptfoo blog. https://www.promptfoo.dev/blog/asr-not-portable-metric/.
- Debenedetti, Edoardo, Jie Zhang, Mislav Balunovic, Luca Beurer-Kellner, Marc Fischer, and Florian Tramer. 2024. "AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents." Advances in Neural Information Processing Systems, Datasets and Benchmarks Track. https://papers.nips.cc/paper_files/paper/2024/hash/97091a5177d8dc64b1da8bf3e1f6fb54-Abstract-Datasets_and_Benchmarks_Track.html.
- Ding, Tianyu, Yaocheng Tong, Muzhi Yang, Fei Jiang, Lei Yang, and Boya Yu. 2026. Re-Executing Coding-Agent Evaluations: Unchanged Decisions and Unresolved Policy Risk. https://arxiv.org/abs/2610.14460.
- Duy, Phan The, Khoa Ngo-Khanh, Nguyen Huu Quyen, and Van-Hau Pham. 2026. PoC-Adapt: Semantic-Aware Automated Vulnerability Reproduction with LLM Multi-Agents and Reinforcement Learning-Driven Adaptive Policy. https://arxiv.org/abs/2604.06618.
- Evtimov, Ivan, Arman Zharmagambetov, Aaron Grattafiori, Chuan Guo, and Kamalika Chaudhuri. 2025. "WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks." Advances in Neural Information Processing Systems, Datasets and Benchmarks Track. https://arxiv.org/abs/2504.18575.
- Gilda, Sankalp. 2026a. Add Adversarial Execution Evidence Predicate (V0.7). In-toto/attestation pull request 570 (draft, not merged). https://github.com/in-toto/attestation/pull/570.
- Gilda, Sankalp. 2026b. Agent-Evidence-Vectors: Conformance Vectors for Agent Execution Evidence. Version 0.30.0. Zenodo. https://doi.org/10.5281/zenodo.23769673.
- Gonuguntla, Ashritha. 2026. The Replay Gap: Static Evaluation of Model Switching in LLM Agents Scores the Wrong World. https://arxiv.org/abs/2608.08239.
- Jia, Haoxuan et al. 2026. The Window Is in the Reasoning: When LLM Agents Decide to Follow Prompt Injections. https://arxiv.org/abs/2610.24048.
- Karamchandani, Neeraj, Piyush Nagasubramaniam, Xinhong Xie, Sencun Zhu, and Dinghao Wu. 2026. Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks. https://arxiv.org/abs/2610.03585.
- Keller, Drew, Kweku Kwegyir-Aggrey, Ryan Steed, Anita K. Rao, Julia L. Sharp, and A. Stevie Bergman. 2026. Expanding the AI Evaluation Toolbox with Statistical Models. https://doi.org/10.6028/NIST.AI.800-3.
- Keller, Drew, Ryan Steed, Tony Wang, A. Stevie Bergman, and Peter Cihon. 2026. Practices for Automated Benchmark Evaluations of Language Models. https://doi.org/10.6028/NIST.AI.800-2.ipd.
- Kirmayr, Johannes, Lukas Stappen, and Elisabeth Andre. 2026. "CAR-Bench: Evaluating the Consistency and Limit-Awareness of LLM Agents Under Real-World Uncertainty." Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 40599-618. https://doi.org/10.18653/v1/2026.acl-long.1886.
- Kumar, Nav. 2026. Offer: A Community-Run AgentDojo Results Space on Hugging Face. Ethz-spylab/agentdojo GitHub issue 248. https://github.com/ethz-spylab/agentdojo/issues/248.
- Li, Alex Chengyu. 2026. AEGIS: A Portable Evidence-Bundle Protocol for Offline Verification of AI-Agent Execution Traces. https://doi.org/10.2139/ssrn.6392459.
- Lim, Soohan, Joonghyuk Hahn, Hyunwoo Park, Sang-Ki Ko, and Yo-Sub Han. 2026. "ContractEval: A Benchmark for Evaluating Contract-Satisfying Assertions in Code Generation." Findings of the Association for Computational Linguistics: ACL 2026, 42549-66. https://doi.org/10.18653/v1/2026.findings-acl.2112.
- Liu, Run. 2026. Some Doubts about the Security Metric. Ethz-spylab/agentdojo GitHub issue 154. https://github.com/ethz-spylab/agentdojo/issues/154.
- Liu, You, and Quanchao Lu. 2026. Replay Fidelity in LLM Agent Evaluation: Randomness, Hidden State, and Test Oracles in AppWorld. https://arxiv.org/abs/2611.11216.
- Liu, Yupei, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. 2024. "Formalizing and Benchmarking Prompt Injection Attacks and Defenses." 33rd USENIX Security Symposium (USENIX Security 24), 1831-47. https://www.usenix.org/conference/usenixsecurity24/presentation/liu-yupei.
- Longpre, Shayne, Kevin Klyman, Ruth E. Appel, et al. 2025. In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI. https://arxiv.org/abs/2503.16861.
- Lu, Xing Han, Amirhossein Kazemnejad, Nicholas Meade, et al. 2025. AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories. https://arxiv.org/abs/2504.08942.
- Luo, Qingzhou, Farah Hariri, Lamyaa Eloussi, and Darko Marinov. 2014. "An Empirical Analysis of Flaky Tests." Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2014), 643-53. https://doi.org/10.1145/2635868.2635920.
- Manski, Charles F. 2003. Partial Identification of Probability Distributions. Springer. https://doi.org/10.1007/b97478.
- Mazeika, Mantas, Long Phan, Xuwang Yin, et al. 2024. HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal. arXiv:2402.04249. https://arxiv.org/abs/2402.04249.
- Mei, Xiang, Jordi Del Castillo, Pulkit Singh Singaria, et al. 2024. ARVO: Atlas of Reproducible Vulnerabilities for Open-Source Software. https://arxiv.org/abs/2408.02153.
- MITRE. 2026. MITRE ATLAS: Adversarial Threat Landscape for AI Systems. Knowledge base, data release v2026.10. https://atlas.mitre.org/.
- MrMoshkovitz. 2025. Bug: MitigationBypass Detector Incorrectly Scores Model Refusals as Failures. NVIDIA/garak GitHub issue 1413. https://github.com/NVIDIA/garak/issues/1413.
- Mu, Dongliang, Alejandro Cuevas, Limin Yang, et al. 2018. "Understanding the Reproducibility of Crowd-Reported Security Vulnerabilities." 27th USENIX Security Symposium (USENIX Security 18), 919-36. https://www.usenix.org/conference/usenixsecurity18/presentation/mu.
- Mudasiru, Rasheed. 2026. Deterministic Replay for AI Agent Systems. https://arxiv.org/abs/2607.16200.
- Nasr, Milad, Nicholas Carlini, Chawin Sitawarin, et al. 2025. The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections. https://arxiv.org/abs/2510.09023.
- Nawaz, Ahad. 2026. Workspace InjectionTask3 Security Check Misses Successful Exfiltration When the Agent Copies the YAML-Rendered Email Body. Ethz-spylab/agentdojo GitHub issue 245. https://github.com/ethz-spylab/agentdojo/issues/245.
- O'Callahan, Robert, Chris Jones, Nathan Froyd, Kyle Huey, Albert Noll, and Nimrod Partush. 2017. "Engineering Record and Replay for Deployability." 2017 USENIX Annual Technical Conference (USENIX ATC 17), 377-89. https://www.usenix.org/conference/atc17/technical-sessions/presentation/ocallahan.
- OASIS Open. 2021. STIX Version 2.1. https://docs.oasis-open.org/cti/stix/v2.1/os/stix-v2.1-os.html.
- OpenTelemetry Authors. 2026a. OpenTelemetry Protocol and Protobuf Definitions. https://github.com/open-telemetry/opentelemetry-proto.
- OpenTelemetry Authors. 2026b. OpenTelemetry Semantic Conventions 1.43.0. https://opentelemetry.io/docs/specs/semconv/.
- Owiredu-Ashley, Harry. 2026. Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents. https://arxiv.org/abs/2607.07474.
- Paduraru, Ciprian, Petru-Liviu Bouruc, and Alin Stefanescu. 2026. A Trace-Based Assurance Framework for Agentic AI Orchestration: Contracts, Testing, and Governance. https://arxiv.org/abs/2603.18096.
- Panickssery, Arjun, Samuel R. Bowman, and Shi Feng. 2024. LLM Evaluators Recognize and Favor Their Own Generations. arXiv:2404.13076. https://arxiv.org/abs/2404.13076.
- Pattnayak, Priyaranjan, and Apoorv Bhatia. 2026. "ReproEvalCard: A Reporting Standard for Reproducible Evaluation of LLM Pipelines." Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 238-49. https://doi.org/10.18653/v1/2026.acl-short.22.
- Piao, Yangheran, Jingjie Li, and Daniel W. Woods. 2025. "Abuse Risks Are Often Inherent to Product Features": Exploring AI Vendors' Bug Bounty and Responsible Disclosure Policies. https://arxiv.org/abs/2509.06136.
- Prabscodes. 2026. Banking Injection Tasks 0/1/3: Operator Precedence Makes the Subject Check Dead Code. Ethz-spylab/agentdojo GitHub issue 181. https://github.com/ethz-spylab/agentdojo/issues/181.
- ProofAgent. 2026. Proofagent-Harness: Open-Source Test Harness for AI Agents. GitHub repository ProofAgent-ai/proofagent-harness, version 0.14.0. https://github.com/ProofAgent-ai/proofagent-harness.
- Qian, Yaoyao, Yuanli Wang, Jinda Zhang, et al. 2025. WebGraphEval: Multi-Turn Trajectory Evaluation for Web Agents Using Graph Representation. https://arxiv.org/abs/2510.19205.
- Rabanser, Stephan, Sayash Kapoor, Peter Kirgis, Kangheng Liu, Saiteja Utpala, and Arvind Narayanan. 2026. Towards a Science of AI Agent Reliability. https://arxiv.org/abs/2602.16666.
- Rajhans, Siddhant. 2026. Latentinjection: Translation Probes Score Faithful Translations and Refusals as Successful Injections. NVIDIA/garak GitHub issue 2451. https://github.com/NVIDIA/garak/issues/2451.
- Raza, Hassan. 2026. LocalLLM Turns Server Errors into an Empty Answer, Which Is Scored as a Failed Attack. Ethz-spylab/agentdojo GitHub issue 220. https://github.com/ethz-spylab/agentdojo/issues/220.
- Rodriguez, Pedro, Joe Barrow, Alexander Miserlis Hoyle, John P. Lalor, Robin Jia, and Jordan Boyd-Graber. 2021. "Evaluation Examples Are Not Equally Informative: How Should That Change NLP Leaderboards?" Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, 4486-503. https://aclanthology.org/2021.acl-long.346/.
- Rundgren, Anders, Bret Jordan, and Samuel Erdtman. 2020. JSON Canonicalization Scheme. RFC 8785. RFC Editor. https://doi.org/10.17487/RFC8785.
- Schwinn, Leo, Moritz Ladenburger, Tim Beyer, Mehrnaz Mofakhami, Gauthier Gidel, and Stephan Günnemann. 2026. A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness. https://arxiv.org/abs/2603.06594.
- SealedRun contributors. 2026. SealedRun: Tamper-Evident, Signed Record of Every AI Agent Step. GitHub repository sealedrun/sealedrun, version 0.10.0. https://github.com/sealedrun/sealedrun.
- Secure Systems Lab. 2023. Dead Simple Signing Envelope Protocol 1.0.2. https://github.com/secure-systems-lab/dsse/blob/master/protocol.md.
- Shah, Jaineet. 2026. Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures. https://arxiv.org/abs/2606.08275.
- Shahriyar, Shaikh Akib, MD. Akil Raihan Iftee, Armon Barton, and Matthew Wright. 2026. What Counts as the Same Task? Executable Contracts for Robustness Evaluation of Tool-Using Agents. https://arxiv.org/abs/2610.14477.
- Siddique, Imran, Rishabh Poddar, and Aaron Fulkerson. 2026. TRACE Specification: Trust, Runtime Attestation, and Compliance Evidence, Version 0.2. Draft (Developer Preview); TRACE Specification, a Series of LF Projects, LLC. https://github.com/agentrust-io/trace-spec.
- Sigstore Authors. 2026. Sigstore Bundle Format 0.3.2. https://docs.sigstore.dev/about/bundle/.
- Solozobov, Oleg. 2026a. Agent-Safety Evaluations as Load-Bearing Evidence: A Vendor-Neutral, Cross-Harness Reconstructability Metric. https://arxiv.org/abs/2607.12469.
- Solozobov, Oleg. 2026b. Decision Evidence Maturity Model for Agentic AI: A Property-Level Method Specification. https://arxiv.org/abs/2605.04093.
- Solozobov, Oleg. 2026c. DEMM-Bench: A Cross-Regime Benchmark for Agent-Runtime Governance-Evidence Sufficiency. https://arxiv.org/abs/2606.20634.
- Souly, Alexandra, Qingyuan Lu, Dillon Bowen, et al. 2024. A StrongREJECT for Empty Jailbreaks. arXiv:2402.10260. https://arxiv.org/abs/2402.10260.
- Tarre, Mark. 2026. OpenAI Launches Safety Bug Bounty for AI Abuse Risks. SecurityBrief. https://securitybrief.news/story/openai-launches-safety-bug-bounty-for-ai-abuse-risks.
- Torres-Arias, Santiago, Hammad Afzali, Trishank Karthik Kuppusamy, Reza Curtmola, and Justin Cappos. 2019. "In-Toto: Providing Farm-to-Table Guarantees for Bits and Bytes." 28th USENIX Security Symposium (USENIX Security 19), 1393-410. https://www.usenix.org/conference/usenixsecurity19/presentation/torres-arias.
- Tsvetkov, Radoslav. 2026. AGEF V0.1.3 Specification: Agent Governance Evidence Format. https://github.com/radotsvetkov/agef.
- UK AI Security Institute. 2026. Inspect AI Evaluation Logs. https://inspect.aisi.org.uk/eval-logs.html.
- van Tonder, Rijnard, John Kotheimer, and Claire Le Goues. 2018. "Semantic Crash Bucketing." Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE '18), 612-22. https://doi.org/10.1145/3238147.3238200.
- Wang, Yiqi, Jiaqi Zhang, Taotao Cai, et al. 2026. From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents. https://arxiv.org/abs/2606.04990.
- Wang, Yongchao, and Zhiqiu Huang. 2026. Security Is Relative: Training-Free Vulnerability Detection via Multi-Agent Behavioral Contract Synthesis. https://arxiv.org/abs/2604.19012.
- Wang, Zhun, Tianneng Shi, Jingxuan He, Matthew Cai, Jialin Zhang, and Dawn Song. 2025. CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale. https://arxiv.org/abs/2506.02548.
- webpro255. 2026. Slack Injection_task_5: Security_from_traces Simulates State from Attempted Calls, Scoring Blocked (Never-Executed) Sequences as Attack Success. Ethz-spylab/agentdojo GitHub issue 168. https://github.com/ethz-spylab/agentdojo/issues/168.
- World Wide Web Consortium. 2013a. PROV-AQ: Provenance Access and Query. https://www.w3.org/TR/prov-aq/.
- World Wide Web Consortium. 2013b. PROV-CONSTRAINTS: Constraints of the PROV Data Model. https://www.w3.org/TR/prov-constraints/.
- Yao, Shunyu, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. 2024. τ-Bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. https://arxiv.org/abs/2406.12045.
- Ying, Jie, Xutao Zhang, Ningyuan Chen, et al. 2026. AgentTriage: Replayable Security Diagnosis Across Heterogeneous Agent Runtimes. https://arxiv.org/abs/2610.30738.
- Zarif, Farhan, Kazi Shahid Ahmed Galib, Synthia Akter Promi, et al. 2026. Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using Agents. https://arxiv.org/abs/2610.23254.
- Zhan, Qiusi, Zhixiang Liang, Zifan Ying, and Daniel Kang. 2024. "InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents." Findings of the Association for Computational Linguistics: ACL 2024, 10471-506. https://doi.org/10.18653/v1/2024.findings-acl.624.
- Zhang, Chuyifei. 2026. When the Reward Suite Is Leaky: A Preregistered Causal Contrast of Natural Verifier False Positives in RLVR. https://arxiv.org/abs/2607.11022.
- Zhang, Hanrong, Jingyuan Huang, Kai Mei, et al. 2025. "Agent Security Bench: Formalizing and Benchmarking Attacks and Defenses in LLM-Based Agents." International Conference on Learning Representations. https://openreview.net/forum?id=V4y0CpX4hK.
- Zheng, Lianmin, Wei-Lin Chiang, Ying Sheng, et al. 2023. "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena." Advances in Neural Information Processing Systems 36 (NeurIPS 2023), Datasets and Benchmarks Track. https://arxiv.org/abs/2306.05685.
- Zhu, Peiying, and Sidi Chang. 2026. ClaimReceipt: Verifying Evidence Sufficiency and Coverage in Agent Evaluations. https://arxiv.org/abs/2609.01992.
- Zhu, Yuxuan, Tengjun Jin, Yada Pruksachatkun, et al. 2025. Establishing Best Practices for Building Rigorous Agentic Benchmarks. https://arxiv.org/abs/2507.02825.