Rate Limiting

Why & where

Rate limiting protects you from abuse, accidental floods, and cost blowups, and enforces fair use / billing tiers. Put it at the edge / API gateway so bad traffic dies before it touches your services. StockVision uses slowapi for this and meters its B2B API keys; StockStump's per-player trade lock is a specialized one-token limiter.

The algorithms

Token bucket is the most common answer: it permits short bursts (up to the bucket size) while bounding the long-run rate.

The two knobs map directly to behavior: R bounds the long-run rate, B bounds the burst. A full bucket after a quiet period is the "controlled burst" — that's a feature, not a bug.

import time

class TokenBucket:
    def __init__(self, rate, capacity):
        self.rate, self.capacity = rate, capacity
        self.tokens = capacity
        self.ts = time.monotonic()

    def allow(self, cost=1):
        now = time.monotonic()
        self.tokens = min(self.capacity, self.tokens + (now - self.ts) * self.rate)
        self.ts = now
        if self.tokens >= cost:
            self.tokens -= cost
            return True
        return False

Here it is running. The bucket starts full, so an opening burst of 5 passes straight through; the 6th request in that burst is refused with a 429, and then the steady drip lets traffic through again. Change R, the capacity B, or the arrival times and watch the burst-versus-throttle trade-off play out:

Distributed rate limiting

With many app servers, an in-memory counter per server multiplies the real limit by the server count. So keep the counter in a shared store (Redis) and mutate it atomically — a Lua script or INCR + EXPIRE makes the read-modify-write a single atomic op, avoiding the same race that bites naive balance checks.

Design drills

You know the five algorithms — now rehearse the system decisions an interviewer actually probes.