Outside the Prompt: Measuring Where AI Coding Agents Trust Repositories, Tools and Connectors

Context: AI coding agents run commands and use tools through the Model Context Protocol (MCP). Security work mostly studies prompt injection. Objective: Measure where disclosed failures of tool-using agents and connectors trigger relative to the model's decision, and how clients behave at configuration and tool boundaries. Method: A preregistered study: classification of 120 stratified advisory-database disclosures of one quarter by the author and three open-weight models; eight clients measured with canary servers; a scored test of default logs; and, exploratory, a trust-dialog study and a census of six official servers. Results: In interactive sessions no client started a repository-declared server before the user answered (one non-interactive mode does, as documented), yet five of six clients that ask pre-select the permissive answer. When an approved tool's description changed mid-session over stdio, no client that asked at first call asked again; one fetched it when notified, two on restart. Per server, 0–97% (median 10.7%) of prose-returning tools pass fully through the server's defence. Default logs missed silent tool changes and server starts in two clients each. With class 3 read broadly, the author placed 90.1% of the frame (95% CI 77.9–95.8%) and the model majority 92.6% outside the model's decision (not robust; Fleiss' κ = 0.46, below the preregistered 0.6); post hoc, blind AI readings of the sources document 63.5% (49.1–74.5%) as such, 33 of 120 unsettled. Conclusions: Several security-relevant decisions lie in product defaults and configuration, outside the model. Evaluations should report client, version and configuration, not only the model.

Online Resource 1 (supplement.pdf): exact client prompts, deviations log D1–D56, protocol details.