Picture an order service with a pool of 200 request threads. Every checkout calls an address-validation API. The API normally answers in 80 milliseconds. One afternoon it starts answering in 30 seconds, because the vendor behind it is having a bad day.
Nothing is down. The API still returns 200s. It just returns them slowly.
At 50 checkouts a second, each holding a thread for 30 seconds, the service needs 1,500 threads and owns 200. The pool is empty in four seconds. The product pages never call address validation, but they share the pool, so they stop answering too. The load balancer's health checks time out, instances leave rotation, and the survivors take their traffic and drown faster.
This is a composite scenario, not one client's outage. Every number in it is arithmetic you can check.
The dependency was slow, not down, and the slow one did more damage. A dead dependency refuses the connection in milliseconds and the thread goes back to the pool. A slow one keeps the thread.
Why Slow Is Worse Than Down
A timeout limits how long one call waits. It does not limit how many calls are waiting at once.
The arithmetic is the one Lambda's documentation uses for concurrency: requests per second times seconds per request (Lambda concurrency). With a 30-second timeout, 50 requests a second need 1,500 threads. Cut the timeout to 5 seconds and you still need 250. Only a timeout of 4 seconds or less keeps the example inside a 200-thread pool, and that is before any retry multiplies the calls (post 1 covers that side of it). This post is part of the Cloud Design Patterns series.
The second problem is the shared pool. When every dependency draws from the same threads or connections, the slowest one sets the capacity of everything. The failure spreads to calls that had nothing to do with it.
Netflix wrote this down when it built Hystrix, its latency and fault-tolerance library. The Hystrix wiki says that under high-volume traffic, a single latent backend dependency can saturate all resources in seconds on all servers. It also describes the remedy: a small pool for each dependency, with requests rejected instead of queued when the pool is full.
AWS Prescriptive Guidance says the same thing in operations terms. Repeated retries against a callee that is timing out can cause network contention and database thread pool consumption, and with many users retrying they make the whole application worse (circuit breaker pattern).
A slow dependency is more dangerous than a dead one. The goal of a circuit breaker is to turn the first into the second on purpose.
How the Pattern Works
A circuit breaker sits between a caller and one dependency and has three states.
Closed. Calls pass through. The breaker counts failures and slow calls over a sliding window. A call that exceeds your latency budget counts as a failure even if it eventually succeeds, because to the thread it was one.
Open. When the failure rate in the window passes a threshold, the breaker opens. Calls now fail immediately, without touching the dependency, or return a fallback. The thread goes back to the pool at once instead of after seconds.
Half-open. After a cool-down, the breaker lets a few probe calls through. If they succeed, it closes. If one fails, it opens again for another cool-down.
Two details decide whether this works.
First, set a minimum number of calls before the breaker may trip. A window with four calls and two failures is a 50% failure rate and tells you nothing. Hystrix's defaults are a reasonable place to start: at least 20 requests in a 10-second rolling window, and a 50% error rate to trip (Hystrix configuration). They are not AWS numbers; tune them from your own traffic.
Second, give each dependency its own small pool or concurrency limit. This is the bulkhead, named for the watertight compartments in a ship's hull. The breaker needs time to see a failure and trip. The bulkhead limits the damage during that time. With 10 concurrent calls allowed to address validation, the worst case in the opening scenario is 10 stuck threads, not 200.
The fallback needs honesty. AWS's own guidance is to keep a degraded path much simpler than the primary one and to test it, and it adds that fallback strategies should generally be avoided (REL05-BP01). The Builders' Library explains why: a fallback is a mode the system enters only when everything else is going wrong, so it is the code least likely to have been exercised (Avoiding fallback in distributed systems). Often the honest answer is a fast, clear error. If you must fall back, use data that is already local; post 8 covers fallbacks that do not call a control plane.
Building It on AWS
AWS has no single managed circuit breaker. In application code you use a library: Resilience4j on the JVM, Polly on .NET, or pybreaker and opossum in Python and Node.js. Their state lives in process memory, which works for a service with long-lived instances. Lambda runs each concurrent request in its own execution environment (Lambda concurrency), so a breaker there needs shared state. Prescriptive Guidance keeps it in DynamoDB, and notes ElastiCache can replace DynamoDB for speed. The bulkhead is the part AWS gives you directly: Lambda reserved concurrency, and Amazon ECS Service Connect's outlier detection. Both are blunter than a library, and neither knows about fallbacks.
One naming trap: the Amazon ECS deployment circuit breaker is a different thing. It rolls back a failed deployment, and it does nothing about a slow dependency at runtime. AWS App Mesh once offered Envoy timeouts and circuit breaking, and it reached end of support on September 30, 2026 (App Mesh notice).
| Service | Setting | Value | Source |
|---|---|---|---|
| AWS Step Functions, Amazon DynamoDB | Orchestrated breaker | An express workflow reads a CircuitStatus table; an unexpired record means open and the workflow fails at once. After retries with exponential backoff fail, it writes a record with an ExpiryTimeStamp, and DynamoDB TTL removes expired items | Circuit breaker pattern |
| AWS Lambda | Reserved concurrency as a bulkhead | Default account limit of 1,000 concurrent executions per Region; 100 always stay unreserved, so 900 can be reserved by default; no charge; 0 throttles a function on purpose | Lambda concurrency, Configuring reserved concurrency |
| Amazon ECS Service Connect | Outlier detection | If 5 or more connections to a task fail in 30 seconds, the proxy avoids that task for 30 to 300 seconds | Service Connect components |
| Amazon ECS Service Connect | Retries and timeout | 2 retries, the second avoiding the previous host; perRequestTimeout defaults to 15 seconds for HTTP, HTTP2, and gRPC | Service Connect components |
| Amazon ECS | Deployment circuit breaker (not a runtime breaker) | Default threshold type BOUNDED_PERCENT at 50% of desired count, clamped to 3 at the low end and 200 at the high end | Deployment circuit breaker |
Capping a payment adapter at 20 concurrent executions takes one command, from the same page (Configuring reserved concurrency):
aws lambda put-function-concurrency \
--function-name payments-adapter \
--reserved-concurrent-executions 20
When Not to Use It
A call with no fallback and no shared resource to protect. A nightly batch job that calls a slow API can simply wait. A breaker there adds code and little else. Put the work behind a queue and let the consumer take its time (post 3).
Low traffic. A breaker tuned on three calls a minute opens on noise. Either set a minimum call count, as above, or skip the breaker and rely on a well-chosen timeout and a retry budget.
One breaker shared across very different failure profiles. If one breaker guards every endpoint of a vendor, or every tenant, one bad endpoint or one noisy tenant opens it for all of them. Key breakers by dependency and operation, and give a tenant with a distinct profile its own.
A caller that is itself overloaded. A breaker protects you from a dependency. It does not shed your own excess load. For that, see load shedding and backpressure.
How We Check for It
In a Well-Architected Review, we look at this under the Reliability pillar, question REL05: how you design interactions in a distributed system to mitigate or withstand failures. The best practices behind it are REL05-BP01 (graceful degradation), REL05-BP04 (fail fast and limit queues), and REL05-BP05 (set client timeouts). In live accounts we look at:
- A dependency inventory for the critical path, naming every synchronous call to a third party or another team's service.
- An explicit connect timeout and request timeout on each call, not a framework default that may be infinite.
- Separate pools or concurrency limits per dependency: reserved concurrency on Lambda functions, Service Connect settings, or code-level bulkheads.
- A tested fallback or a deliberate fast failure for each dependency, with the behavior written down.
- Breaker state changes emitted as metrics, with an alarm on a breaker that stays open.
These checks are part of the AWS Well-Architected Framework Review.
Related Patterns
- Retries Without Retry Storms: the breaker is what stops a retry loop against a dependency that is not coming back.
- Load Shedding and Backpressure: protects you when the problem is your own traffic, not a dependency.
- Static Stability: fallbacks that work without calling a control plane.
- Observability With RED, USE, Traces, and SLOs: how you see a breaker open before a customer tells you.