Agent Failure of the Week #1: The Thundering Herd

The failure

A team ships an enrichment agent. In testing, with one agent and a handful of records, it's flawless. They roll it out across the full lead queue and add a second agent for routing and a third for summarization. Within an hour, leads start landing in the wrong territories, enrichment fields come back blank, and a handful of opportunities get created twice. Nothing in the logs says "error." The agents all reported success.

What actually happened: the three agents, running in parallel across thousands of records, generated more API calls in ten minutes than the sales team generates in a week. The CRM's rate limiter started throttling requests. The agents, which had no concept of "wait and retry," read each throttled call as an empty result (no enrichment data, no matching territory) and confidently acted on the void.

Why your stack wasn't built for this

A human user is, from your infrastructure's perspective, almost perfectly idle. They open one record, read it, think, save. Even a busy team is a trickle. Every rate limit, connection pool, and timeout in your stack was sized, implicitly or explicitly, against that trickle.

Agents are not a trickle. A single agentic workflow fans out into dozens of concurrent calls, and a fleet multiplies that. You don't ramp up gently; you go from human-scale traffic to machine-scale traffic the moment you flip the switch. The stack hits limits nobody knew existed, and because agent workflows are chained, each step feeding the next, one throttled call doesn't fail alone. It cascades downstream through every workflow that depended on it.

The test (how we score it)

Do your APIs support multiple agents calling simultaneously?

• Green: Load-tested against realistic concurrent agent traffic; rate limits documented; burst handling (backoff, queuing) designed in.
• Yellow: Limits are known but never tested under agent-scale concurrency.
• Red: Unknown. Never tested. You'll find the ceiling in production.

The fix

1. Load-test before you deploy, not after. Simulate the concurrent call volume of your actual agent fleet: not one agent, the whole herd at peak.
2. Make limits explicit and visible. Document every rate limit on every API an agent touches, internal and SaaS.
3. Build backoff and queuing into the agent layer. An agent that hits a limit should slow down and retry, never interpret a throttle as an empty result.
4. Degrade gracefully. Under pressure the system should slow down, not silently drop work and cascade.

The bigger point

This isn't an AI problem. It's a concurrency problem, the kind of thing load testing has caught for decades. Agents are just the first actor ruthless enough to find your ceiling instantly, at scale, without the human instinct to ease off. Get the foundation right and a fleet of agents is a force multiplier. Get it wrong and it's a cascade waiting for a trigger.

This is exactly what we test on the Agentic Workflow Readiness pillar of our AI Readiness Benchmark, before your agents find the ceiling for you.

See where your stack stands → revenuegroundwork.com/benchmark Or book a free consultation → revenuegroundwork.com/contact

Next week, #2: two agents write to the same record at the same time, and your stack has no opinion about who wins.