The System Design Interview · How it works
8 min readHow it works
Why "design Twitter" is not yet a question
"Design Twitter" is a prompt, not a problem. Twitter has direct messages, ads, search, trending topics, video transcoding, and a recommendation engine — nobody designs that in 45 minutes. The interviewer's actual question is closer to "show me you can choose a slice, justify the slice, and reason about it precisely."
So the first move is always to narrow. You are not stalling; you are doing the part of the job that has the highest leverage. A design that cleanly handles posting a tweet and reading a timeline is a strong answer. A shallow diagram touching eleven features is a weak one, because nothing in it can be examined.
A: Because every box you draw is an answer to a question you have not asked yet. Without knowing the read/write ratio you cannot know whether to denormalise; without knowing whether staleness is acceptable you cannot know whether a cache is legitimate or a correctness bug; without a scale estimate you cannot know whether one database is fine. Drawing early commits you to a shape you will then defend by reflex rather than by reasoning, and interviewers read that as pattern-matching rather than engineering. The stronger opening is to establish constraints, because the constraints determine the design — and if you get the constraints on the board, a competent design often follows almost mechanically.
Functional versus non-functional requirements
Functional requirements are what the system does — the verbs. Non-functional requirements are the qualities it must hold while doing it, and these are the ones that actually drive architecture.
| Scale | How many users, and how many actions each per day? | Decides whether one machine is enough |
|---|---|---|
| Read/write ratio | Is this 100:1 read-heavy or write-heavy? | Read-heavy justifies caching and replicas; write-heavy forces sharding |
| Latency | What is acceptable at p99, not average? | Sub-100ms rules out cross-region synchronous calls |
| Consistency | Is stale data acceptable, and for how long? | Decides replication strategy and whether caching is safe |
| Durability | Is losing the last second of writes acceptable? | Decides synchronous versus asynchronous persistence |
| Availability | What is the cost of an hour of downtime? | Decides redundancy, multi-region, failover complexity |
The trap in this list is latency. Candidates say "low latency" and move on. Averages hide everything: a system where 99% of requests take 10ms and 1% take 3 seconds has an excellent average and a terrible user experience, because at scale that 1% is millions of people, and a single page issuing twenty backend calls will hit the slow path more often than not.
A: Because the average is dominated by the fast majority and hides the tail, and the tail is what users actually experience. If one request in a hundred takes three seconds, the average barely moves — but a page that fans out to twenty backend calls has roughly an 18% chance of hitting at least one slow call, so the slow path becomes the common path for whole page loads rather than a rare event. Tail latency also compounds through layers: each hop adds its own tail, so a request crossing five services inherits five chances to be slow. That is why the meaningful target is a percentile, and why the answer to "how do we make this faster" is often "find and fix the tail" rather than "reduce the mean".
Numbers you should be able to produce from memory
Estimation is not about being right. It is about being approximately right quickly, and showing that you know which quantity matters. Round aggressively — the point is the order of magnitude.
A useful anchor set:
- A day is ~86,400 seconds. Call it 100,000 — the error is 15% and the arithmetic gets ten times easier.
- One million writes a day is therefore roughly 12 per second. This is small.
- One billion writes a day is roughly 12,000 per second. This is not small.
- A single well-indexed relational database handles thousands of simple reads per second and low thousands of writes per second before tuning becomes a project.
- Memory access is ~100 nanoseconds; an SSD read is ~100 microseconds (1,000× slower); a cross-continent round trip is ~150 milliseconds (again ~1,000× slower).
- A UUID is 16 bytes; a timestamp 8; a short text field tens of bytes. A "row" of metadata is usually ~100 bytes to 1 KB.
From those you can derive the two numbers that matter most: requests per second, and bytes per day.
-- Estimation is often just a SELECT you never run. Sizing a URL-shortener:
-- 1 billion new links/year, each row: id(8) + code(8) + url(200) + user(16) + ts(8)
-- ~240 bytes -> ~240 GB/year of raw rows, before indexes.
CREATE TABLE short_links (
id BIGINT PRIMARY KEY,
code VARCHAR(8) NOT NULL,
target_url VARCHAR(2048) NOT NULL,
owner_id UUID NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE UNIQUE INDEX idx_short_links_code ON short_links (code);That index matters more than the table: the read path is entirely "given a code, find the URL", so the code column must be unique and indexed, and the table itself is almost never scanned.
A: That is one billion requests per day. Using 100,000 seconds per day rather than 86,400 for easier arithmetic, that is 10,000 requests per second average. But average is the wrong number to design against, because traffic is not uniform — real consumer traffic typically peaks at two to three times the daily average within the busiest hours, so I would plan for roughly 20,000 to 30,000 requests per second and confirm the multiplier with the interviewer. I would then immediately convert that into the question that matters: what share is reads versus writes, because 30,000 reads per second is a caching and read-replica problem while 30,000 writes per second is a sharding problem, and those are entirely different designs.
Design the simple thing first, then break it on purpose
The strongest pattern is to propose the boring architecture — a load balancer, a stateless application tier, one database — and then interrogate it out loud. State where it breaks and at what number, and scale only that.
This works because it makes your reasoning legible. "I'll add a cache" is a guess. "Reads are 100× writes and the same 10,000 items are hot, so the database is doing redundant work; a cache in front of the read path removes most of it, at the cost of staleness which is acceptable here because a profile view can be a few seconds old" is engineering.
# The boring first cut, stated concretely enough to critique.
apiVersion: apps/v1
kind: Deployment
metadata:
name: api
spec:
replicas: 3 # stateless -> horizontal scaling is just this number
template:
spec:
containers:
- name: api
image: api:v1
readinessProbe:
httpGet:
path: /healthz
port: 8080
resources:
requests:
cpu: 250m
memory: 256MiThe reason the application tier is drawn stateless is not tidiness — it is that replicas: 3 only works as a scaling lever if no request depends on which instance receives it. Any session state held in process memory silently converts horizontal scaling into a correctness bug.
A: A stateless service keeps no per-client data in its own memory or disk between requests, so any instance can serve any request and instances are interchangeable. It matters because it is what makes horizontal scaling and failure recovery trivial: you can add instances behind a load balancer without coordination, remove them without draining anything meaningful, and a crash loses nothing but the in-flight requests. The moment a service keeps session data in local memory, the load balancer has to route each client back to the same instance — sticky sessions — and now scaling is uneven, one instance failing logs those users out, and deploys become disruptive. The state has not disappeared, it has moved: to a shared store like Redis, or to a signed token held by the client.
Reading the interviewer
The interviewer is usually steering toward one or two specific areas of depth, and their questions are the signal. If they keep asking about the write path, they want to discuss sharding or ordering. If they ask "what happens if that node dies", they want failure handling, not more features. Answer the question they asked, then offer the next layer — do not deliver a monologue that runs past the thing they were probing.
Saying "I don't know, but here is how I'd find out" is a strong answer. Inventing a confident wrong mechanism is the weakest one available, because it also removes the interviewer's ability to trust anything else you said.
Quick recall
Short-answer versions of the same material, for spaced repetition.
A: Scale in users and actions per day, the read-to-write ratio, the p99 latency target, and the tolerable staleness. Together they decide whether you need caching, replicas, or sharding.
A: About 86,400, rounded to 100,000 so the arithmetic is trivial. One million writes a day is then roughly 12 per second.
A: Recovery Point Objective: how much data you can afford to lose, measured in time. Zero for synchronous replication, up to the replication lag for asynchronous.
A: Recovery Time Objective: how long recovery may take before service is restored. It is the target for failover and promotion, not for data loss.