Why Your MCP Server Stops Scaling, and What Actually Fixes It
Most MCP scaling failures are not capacity problems. They are per-client session state that prevents horizontal scaling, a tool surface that consumes client context on every request, and responses sized for a human reader that overwhelm the caller. Fix statelessness, shrink the surface, and paginate before you add servers.
The three walls, in the order teams hit them
Session state. Early servers hold context per connection: an authenticated session, a cursor, an open handle, a cached working set. It is convenient and it means any given client must keep talking to the same process. Now you need sticky routing, your deploys drop live sessions, and a restart is a visible failure rather than a rolling one. This is the wall that forces an architecture change rather than a configuration change.
Tool surface. Every tool you expose is serialized into the client's context on every single request for the whole session, used or not. A server with thirty verbose tool descriptions is charging a tax to every conversation that connects to it, and it degrades tool selection across everything else the client has connected. This cost is invisible on your side, which is why it goes unnoticed until a client complains that the agent got worse after adding your integration.
Response size. A tool that returns a full record, a whole file, or an unbounded list works beautifully in testing and consumes the caller's window in production. Your metrics look fine. The client's agent starts failing.
Notice that two of the three failures land on the client, which is exactly why they are diagnosed late.
Make it stateless, then make it boring
The fix for the first wall is standard service design applied to a protocol people are still treating as special.
Push state to the client or to a shared store. If a call needs context from a previous call, that context should arrive in the arguments or be resolvable from a token the client holds. Authentication resolves per request rather than per connection.
Once a server is stateless, everything ordinary becomes available: horizontal scaling, rolling deploys without dropping sessions, health checks that mean something, and the ability to restart a process without anyone noticing.
Two details specific to agent traffic. First, idempotency matters earlier than with human clients, because agents retry on timeouts and ambiguous errors far more aggressively than people do, and a retried write without deduplication is a duplicate record. Second, rate limits should be per caller identity rather than per connection, since a single agent can open many connections and a single connection can carry an enormous burst.
Cache the expensive reads
Agent traffic is unusually repetitive. The same lookups recur across sessions and across users, because agents re-orient constantly. A short cache in front of expensive reads absorbs a surprising share of load, and unlike human traffic the repetition is predictable enough that a simple key on the arguments works well.
Shrink the surface and bound the payload
The second and third walls are design work rather than infrastructure.
On surface: expose the smallest set of tools that covers the tasks agents actually attempt, and describe each precisely enough that selection is unambiguous. Merge tools that differ only by a parameter. Delete the ones added speculatively. If a client connects your server alongside four others, your thirty tools are not just your cost, they are a shared degradation.
On payloads: return what the caller needs to make the next decision, not everything you know. Default to a summary with an identifier the agent can use to fetch more, paginate anything unbounded, and set a hard maximum on response size rather than trusting that the query will be narrow.
And say what was truncated. An agent that receives a silently trimmed list will reason confidently about a partial answer, which is worse than an error, because nothing signals that anything went wrong.
When more capacity is the wrong answer
Before scaling out, look at what your calls actually do.
If most of them perform an action, changing state in a system the agent cannot otherwise reach, then a server is the right shape and the fixes above apply.
If most of them return documents, references, or knowledge, the load is a symptom rather than the problem. A tool wrapping a document store makes the model responsible for deciding when to look, and models under-call optional lookup tools because nothing in the conversation reminds them the material exists. So agents either call constantly, which is your load problem, or too rarely, which is a quality problem, and neither is fixed by more servers.
Knowledge belongs in an index that is searched before generation rather than in a tool the model must remember to invoke. RDK handles that layer: files from local vaults, docs, and code are indexed as encrypted private chunks, and agents search those chunks before querying a model. Token spend drops 80 to 90 percent on repeated or reference-heavy work because the answer is retrieved instead of regenerated, and the load never reaches your server at all.
Build the server for access. Index for knowledge. Most scaling pain in this space comes from asking one component to do both.
Frequently asked questions
- Why does my MCP server not scale?
- Usually per-connection state rather than capacity. A server holding session context requires sticky routing, drops live sessions on deploy, and cannot scale horizontally. Push state into arguments or a shared store and resolve authentication per request, and ordinary scaling techniques become available again.
- How many tools should an MCP server expose?
- As few as cover the tasks agents attempt. Every tool is serialized into the client's context on every request for the whole session, so a large surface taxes every conversation that connects and degrades selection across other servers too. Merge tools differing only by a parameter and delete speculative ones.
- How large should MCP tool responses be?
- Bounded, and sized for a decision rather than for completeness. Return a summary plus an identifier the caller can use to fetch detail, paginate anything unbounded, and enforce a hard maximum. Always state what was truncated, since an agent given a silently trimmed list reasons confidently about a partial answer.
- When is an MCP server the wrong solution?
- When most calls return documents or knowledge rather than performing actions. A tool wrapping a document store makes the model responsible for remembering to look, and models under-call optional lookup tools. Index that content and retrieve it before generation instead, which removes the load rather than serving it.