The MCP Tool Trap: Why More Tools Make AI Dumber

Episode summary: As AI agents connect to more tools, they can drown in the data required to use them. This episode explores the Model Context Protocol's context pollution crisis and how just-in-time tool usage solves it. Learn how dynamic discovery and caching can slash token usage by 90% and restore reasoning speed, turning a sluggish assistant into a snappy one.

Show Notes

AI agents are getting more powerful, but they are also getting bogged down. The problem is not a lack of capability, but a surplus of it. When an agent connects to multiple tools—like GitHub, Slack, or a local filesystem—it must load the description of every available tool into its context window before it can even start working. This "context bloat" can consume tens of thousands of tokens, leaving little room for the actual conversation or reasoning. The result is a sluggish, confused agent that forgets its purpose after scanning a massive list of JSON schemas.

The core issue is known as the MCP tool trap. The Model Context Protocol (MCP) is a standard for connecting models to local data and APIs, but the more tools an agent has, the more its performance degrades. One developer reported that loading 50 tools used 80% of their context window on startup. Another benchmark showed that loading 400 tools statically would require over 400,000 tokens—more than most models can handle in a single turn. This is not just a cost issue; it is a reasoning issue. When a model has to sift through a haystack of tool definitions, latency increases, hallucinations become more likely, and the model loses the thread of the conversation.

The solution is just-in-time (JIT) tool usage. Instead of loading all tools upfront, the agent only fetches the schema for a tool when it actually needs it. This is achieved through a "discovery phase" where the agent uses a meta-tool—a tool for finding tools. The agent starts with a lightweight list of tool names and descriptions, then uses semantic search (like RAG for tools) to find the right one for the task. Once the intent is confirmed, the system injects the full JSON schema. This approach can reduce context usage from 80,000 tokens to just 6,000—a hundred-fold reduction.

JIT tool usage also changes how developers design tools. Instead of creating massive, Swiss Army knife tools to keep the count low, developers can build atomic, hyper-specific functions. This aligns with the Unix philosophy: do one thing and do it well. With JIT, the agent can handle thousands of tiny tools, pulling them down like npm packages on the fly. Caching is key to managing latency. Once a tool's schema is fetched, it is kept in an in-memory cache for the session, so the "discovery tax" is paid only once.

Standards are evolving to support this. The January 2026 MCP update formalized tool discovery, allowing servers to expose metadata without sending the full schema. Open-source projects like the MCP tool router have pioneered local caching layers that predict which tools will be needed based on common workflows. Cloud-native registries, like Composio's AI control plane, offer a centralized discovery layer that vectorizes thousands of APIs, though privacy concerns may favor local implementations for sensitive environments.

Ultimately, JIT tool usage uncaps the number of tools an agent can use. It moves the industry from a "rationing era" to a "search to use" flow, where agents can access vast toolkits without drowning in context. For developers, this means more granular tools, better reasoning, and lower costs. The future of AI agents is not about having fewer tools, but about using them smarter.

Listen online: https://myweirdprompts.com/episode/mcp-tool-trap-context-bloat