Rate limiting & resilience Study Portal

A: Because IP addresses do not map to users. Many users share one address behind corporate NAT, carrier-grade NAT or a proxy, so a per-IP limit throttles innocent users together; meanwhile an attacker with a botnet or rotating cloud addresses gets a fresh limit per address. IP limits are useful as a coarse edge defence against floods from unauthenticated traffic, but fairness and abuse control should key on an authenticated identity — API key, user or tenant.

A: A client can send the full quota at the very end of one window and again at the very start of the next, so within a few seconds it sends twice the intended rate while never exceeding the per-window count. With a limit of 100 per minute, 100 requests at 12:00:59 and 100 at 12:01:00 are both allowed. Sliding windows fix it by counting over the last 60 seconds from now rather than per clock minute, and a token bucket fixes it by bounding bursts to the bucket size.

A: The bucket holds up to B tokens and refills at r tokens per second. A client that has been quiet accumulates tokens up to B and can spend them all at once — a burst of B requests — but after that it can only proceed at the refill rate r. Over any long interval it cannot exceed B plus r times the interval, so the average is bounded by r. Setting B controls how bursty clients may be, and r controls sustained throughput.

A: Store the per-key state centrally, usually in Redis, and make the check-and-update atomic — a Lua script that refills a token bucket and takes a token in one round trip, or a sliding window counter with INCR and expiry — so concurrent instances cannot both take the last token. Keep the store close to the gateways, and decide explicitly what happens if it is unreachable: fail open (allow traffic, protecting availability) for general APIs, fail closed for expensive or abuse-prone endpoints. If the extra hop is too costly, have each instance count locally and sync with the store every few hundred milliseconds, accepting a small overshoot.

A: 429 means this client has exceeded its own allowance — the problem is the caller's request rate, and other clients are unaffected. 503 means the service is temporarily unable to handle requests at all, regardless of who sent them, typically because it is overloaded or in maintenance. Both should carry Retry-After. The distinction matters for client behaviour and for monitoring: a spike of 429s is abuse or a misbehaving client, a spike of 503s is a capacity or health problem.

A: They multiply. If each of three layers retries a failed call three times, one user request can produce 3 × 3 × 3 = 27 attempts against the bottom dependency — exactly when it is already struggling. The extra load deepens the slowdown, which causes more timeouts and more retries. Fix it by retrying at one layer only (usually closest to the user or the failing call, not both), using retry budgets that cap retries to a percentage of normal traffic, backing off with jitter, and only retrying idempotent and transient failures.

A: Because the caller stops waiting at its own deadline, and any work still running below it after that is wasted. If the edge gives up after 2 seconds but a backend call has a 5-second timeout, the backend keeps a thread and a connection busy for 3 seconds serving nobody, reducing capacity exactly when the system is slow. Setting each hop's timeout inside the remaining budget — ideally by propagating a deadline with the request — frees resources as soon as the answer can no longer be used.

A: Timeouts bound how long each call waits, but a caller hitting a dead dependency still waits the full timeout on every request and keeps sending it load; retries make that worse. A circuit breaker notices the failure rate is high and stops calling altogether for a cool-down period, so the caller fails in microseconds instead of seconds (keeping its threads free and its own latency low), the dependency gets breathing room to recover, and a half-open probe detects recovery automatically.

A: Isolating resources so that one failing part cannot consume the capacity others need. Example: a product page calls pricing, inventory and recommendations. With one shared pool of 200 HTTP connections, a hanging recommendations service can hold all 200 and take pricing down with it. With a separate pool per dependency — say 100 for pricing, 60 for inventory, 40 for recommendations — a recommendations hang exhausts only its 40, and the page renders without recommendations.

A: It hides overload until it is catastrophic. Work keeps being accepted, the queue grows, every item waits longer, and by the time items are processed the users who sent them have given up — so the service spends its capacity on requests nobody is waiting for, while memory grows until the process dies. A bounded queue with rejection (backpressure or load shedding) keeps latency predictable for the requests it accepts and tells callers immediately to back off or try elsewhere.