Disney World on a summer morning. Thousands of families rushing toward gates. Without control, it would be a stampede. Enter the turnstiles: mechanical devices ensuring only one person passes at a time, controlling flow from flood to manageable stream.
That’s rate limiting: the turnstiles that prevent your service from being trampled by unlimited requests.
Without Turnstiles
Human Stampede
Gates open at 9 AM:
- 10,000 people surge forward
- No control mechanism
- Crushing at entrances
- Staff overwhelmed
- Injuries inevitable
- Park systems crash
Pure demand meets finite capacity. Disaster.
Cascade Breakdown
Even if people get in:
- Ticket scanners overloaded
- Lines form everywhere
- Rides can’t handle load
- Food services swamped
- Total breakdown
Success becomes failure through volume.
Strategies
Fixed Window
“100 entries per hour”
- 9:00-10:00: 3,000 allowed
- 10:00-11:00: 3,000 allowed
- Sharp reset each hour
Problem: 9:59 has 2,999 entries, 10:00 counter resets, 3,000 more enter, 6,000 in 2 minutes.
Sliding Window
“100 entries per rolling hour”
Track last 60 minutes continuously. Always enforce limit. No reset spikes. Smooth flow.
Token Bucket
- Bucket holds 100 tokens
- Refills at 10 tokens/minute
- Entry costs 1 token
- No token? Wait or leave
Natural burst handling with sustained rate control.
Leaky Bucket
- Queue entries
- Process at constant rate
- Overflow rejected
- Smooth output
This diagram requires JavaScript.
Enable JavaScript in your browser to use this feature.
Common Problems
Thundering Herd
Concert tickets on sale:
- 10 AM: Sales open
- 1 million try simultaneously
- Rate limiter overwhelmed
- System still crashes
Even turnstiles have limits.
Solutions: Queue before the queue, lottery system, staggered access.
Retry Storm
Rejected visitors keep trying:
- First attempt: Rejected
- Retry immediately: Rejected
- Multiply by thousands
- Self-inflicted DoS
Solutions: Exponential backoff, clear retry times, virtual queue numbers.
Unfair Distribution
Aggressive users dominate:
- Scripts checking every second
- Regular users can’t get in
- Bots consume capacity
Solutions: Per-user limits, CAPTCHA challenges, behavior analysis.
Decision Rules
Communicate clearly: current limits, reset timing, retry guidance, status codes.
Be generous when possible: start with reasonable limits, adjust based on data.
Rate limiting transforms chaotic demand into manageable flow. The key is finding the right balance: too restrictive frustrates users unnecessarily, too permissive risks system collapse.