System Design Roadmap
System design is the skill of shaping software that keeps working as users, data and teams grow. It covers how requests move through load balancers and services, where data is stored and copied, how caches and queues absorb load, and how the whole thing survives failures. Back-end, full-stack, platform and senior engineers use it every day, and most mid-level and senior interviews include a design round.
This roadmap starts with the core trade-offs and networking basics, then builds up the parts of a scalable system: the application tier, databases, caching and messaging. Later stages cover consistency, availability, the cloud design patterns for resilience and security, performance antipatterns and monitoring. It ends with the classic interview designs and a method for practising them.
By the end you will be able to run a 45-minute design interview from requirements to deep dive, estimate load, choose components and defend each trade-off. Practise designs in system design practice and track progress in progress page.
- Level
- Intermediate to advanced
- Time
- 14–20 weeks at about 8 hours a week
- Scope
- 60 topics in 10 stages
- Before you start
- Experience building at least one back-end service with a database; Basic SQL Roadmap and HTTP knowledge; Familiarity with Computer Science Roadmap basics such as processes, networking and hashing
What you will be able to do
- Run a structured design interview: requirements, estimates, API, data model, architecture and deep dive.
- Estimate traffic, storage and bandwidth and use the numbers to choose an architecture.
- Scale stateless services with load balancers, CDNs and autoscaling.
- Choose and scale databases with indexes, replication, federation and sharding.
- Design caching and messaging layers with clear consistency and failure behaviour.
- Apply resiliency patterns such as retries with backoff, circuit breakers, bulkheads and throttling.
- Design classic systems such as a URL shortener, rate limiter, news feed and chat service.
The roadmap at a glance
- Stage 1Introduction and Core Trade-offsBeginner · 1–2 weeks · 6 topics
- Stage 2Networking and CommunicationBeginner · 1–2 weeks · 6 topics
- Stage 3Scaling the Application TierIntermediate · 2 weeks · 7 topics
- Stage 4Databases and StorageIntermediate · 3 weeks · 6 topics
- Stage 5CachingIntermediate · 1–2 weeks · 4 topics
- Stage 6Asynchronism and MessagingIntermediate · 2–3 weeks · 8 topics
- Stage 7Consistency, Availability and Distributed DataAdvanced · 2 weeks · 4 topics
- Stage 8Reliability Patterns and Cloud Design PatternsAdvanced · 2–3 weeks · 9 topics
- Stage 9Performance Antipatterns and MonitoringAdvanced · 1–2 weeks · 4 topics
- Stage 10Classic Designs and Interview PracticeAdvanced · 3–4 weeks, then ongoing · 6 topics
Stage 1 Introduction and Core Trade-offs
Goal. Explain what a design interview tests and reason about the basic trade-offs every design rests on.
What is System Design? Core
System design is deciding how the parts of a software system fit together so it meets its goals: which services exist, where data lives, how requests travel, and what happens when traffic grows or a machine dies. Code-level design asks how a class should look; system design asks how many servers, which database, which cache, and how they talk.
Interviews for mid-level and senior roles almost always include a design round. There is no single right answer. The interviewer watches how you gather requirements, propose a reasonable architecture, spot bottlenecks and justify each choice with a trade-off. This roadmap is the introduction, the building blocks and the patterns, followed by the classic designs you should rehearse.
Practice. Pick an app you use daily and sketch, in five boxes, how a request from your phone reaches its data.
How to Approach System Design? Core
Use the same structure every time so you never freeze:
- Requirements: list functional features, then non-functional goals such as latency, availability and consistency. Agree what is out of scope.
- Estimates: users, requests per second, storage per year, read to write ratio.
- API and data model: the main endpoints and the core tables or documents.
- High-level design: clients, load balancer, services, databases, caches, queues.
- Deep dive: the hardest part, such as the feed, the counter or the matching logic.
- Bottlenecks and failures: what breaks first, how you scale it and how you recover.
Talk while you draw, check in with the interviewer, and keep an eye on time.
Practice. Run through all six steps for a pastebin service in 35 minutes, out loud, with a timer.
Back-of-the-Envelope Estimation Core
Rough numbers decide the architecture. If 10 million daily users each make 20 requests, that is 200 million a day, about 2,300 per second on average and perhaps five times that at peak. If each write stores 1 KB and there are 50 million writes a day, you add about 50 GB a day, or roughly 18 TB a year before replication.
Useful habits: a day has about 86,400 seconds (round to 100,000), memory reads are nanoseconds, a same-region network round trip is under a millisecond, a cross-continent one is tens to over a hundred milliseconds, and a disk seek is milliseconds. Round freely; the goal is the order of magnitude, which tells you whether one database is enough or you must shard.
Practice. Estimate peak requests per second and five-year storage for a photo sharing app with 5 million daily uploads.
Performance vs Scalability Core
A performance problem means the system is slow for a single user, even with no load. A scalability problem appears only as load grows: one user is served quickly, but a thousand at once cause slowdowns or errors. The fixes differ: performance problems need better algorithms, indexes or caching; scalability problems need the ability to add resources and spread work across them.
A scalable system gets proportionally more throughput when you add hardware. That depends on avoiding shared bottlenecks such as one database, a global lock or sticky in-memory sessions. Say which problem you are solving before proposing a fix.
Practice. For a slow page you know, decide whether the cause is performance or scalability and justify it.
Latency vs Throughput Core
Latency is how long one operation takes, such as 120 ms for an API call. Throughput is how many operations finish per unit of time, such as 5,000 requests per second. You usually want the highest throughput you can get while keeping latency within a target.
Report latency as percentiles, not averages: p50 shows the typical user, p99 shows the slow tail that large systems hit constantly, because one page may call dozens of services. Batching raises throughput but adds latency; parallel calls lower latency but cost more resources. Queues hide latency from users by moving work out of the request path.
Practice. Measure p50 and p99 latency of an endpoint with a load tool at three different request rates.
Availability vs Consistency and the CAP Theorem Core
In a distributed data store, a network partition can split nodes so they cannot talk. The CAP Theorem says that during a partition you must choose: keep answering, possibly with stale data (availability), or refuse some requests until nodes agree (consistency). Since partitions will happen, the real choice is CP or AP behaviour when they do.
A bank balance usually favours consistency; a social like counter favours availability. PACELC extends the idea: even without a partition, you trade latency against consistency, because waiting for replicas to confirm takes time. In interviews, choose per feature rather than for the whole system.
Practice. For a ride-hailing app, label each feature (payments, driver location, trip history) as CP or AP and explain why.
Stage 2 Networking and Communication
Goal. Explain how a request travels across the network and choose a protocol and API style for each interaction.
Domain Name System Core
DNS turns a name such as api.example.com into an IP address. The client asks a resolver, which walks from the root servers to the top-level domain servers to the domain's authoritative servers, and caches the answer for its time-to-live (TTL). Record types to know: A and AAAA (addresses), CNAME (alias), MX (mail), TXT (verification) and NS (delegation).
DNS is also a scaling tool. Weighted or latency-based records send users to the nearest region, and health-checked records remove a failed region. Low TTLs allow fast failover but increase lookup traffic; clients and resolvers may ignore TTLs, so DNS failover is never instant.
Practice. Use dig to trace the full resolution of a large website and note each TTL.
TCP and UDP Core
TCP gives a reliable, ordered byte stream: a handshake opens the connection, lost packets are resent and flow control stops a fast sender from flooding a slow receiver. That reliability costs round trips and head-of-line blocking, where one lost packet delays everything behind it.
UDP sends independent datagrams with no delivery or order guarantee and no connection set-up. It suits live video, voice, games and DNS lookups, where late data is useless and the application can tolerate or repair loss. QUIC, which carries HTTP/3, builds reliable streams on top of UDP to avoid TCP's head-of-line blocking and to speed up connection set-up.
Practice. List five features of an app you know and decide whether each would run over TCP or UDP.
HTTP Core
HTTP is the request-response protocol of the web and most APIs. A request has a method (GET, POST, PUT, PATCH, DELETE), a path, headers and an optional body; the response has a status code, headers and a body. Know the status families: 2xx success, 3xx redirect, 4xx client error (400, 401, 403, 404, 409, 429), 5xx server error (500, 502, 503, 504).
HTTP/1.1 reuses connections with keep-alive. HTTP/2 multiplexes many requests on one connection with compressed headers. HTTP/3 runs over QUIC. Headers such as Cache-Control and ETag drive caching, and TLS (HTTPS) is expected everywhere.
Practice. Inspect a page load in browser developer tools and explain five response headers you see.
REST, RPC, gRPC and GraphQL Core
Four common API styles, each with a natural home:
- REST: resources addressed by URLs and manipulated with HTTP methods. Simple, cache friendly and universal; the default for public APIs.
- RPC: call a remote function as if it were local, such as
getUser(42). Clear for internal actions that are not natural resources. - gRPC: an RPC framework using Protocol Buffers over HTTP/2, with generated clients, streaming and small binary payloads. Popular between internal services.
- GraphQL: one endpoint where the client asks for exactly the fields it needs. Good for varied front ends; harder to cache and needs limits on query cost.
In interviews, REST for public APIs and gRPC between services is a safe answer. See the API Design Roadmap roadmap for depth.
Practice. Design the same "get order with items" call in REST, gRPC and GraphQL and compare payloads.
Real-Time Communication Recommended
Chat, live scores and notifications need the server to push data. Short polling asks for updates every few seconds, which is simple but wasteful. Long polling holds the request open until there is news, then the client reconnects. Server-Sent Events keep one HTTP response open for a one-way stream from server to client.
WebSockets upgrade an HTTP connection into a two-way channel that stays open, which suits chat and multiplayer apps. Each open connection uses memory on a server, so large chat systems run dedicated connection servers and a lookup service that records which server holds each user.
Practice. Build a tiny notification feed twice, once with long polling and once with Server-Sent Events.
Idempotent Operations Core
An operation is idempotent if repeating it has the same effect as doing it once. GET, PUT and DELETE are meant to be idempotent; POST is not. This matters because networks fail mid-request: the client does not know whether the server acted, so it retries, and a non-idempotent payment could run twice.
The standard fix is an idempotency key: the client sends a unique ID with the request, the server stores the result under that key and returns the stored result for any repeat. Message consumers need the same care, because most queues deliver at least once.
Practice. Add an idempotency key to a "create payment" endpoint and prove a retried request does not charge twice.
Stage 3 Scaling the Application Tier
Goal. Scale stateless services horizontally behind load balancers and serve static content from the edge.
Horizontal Scaling Core
Vertical scaling buys a bigger machine: easy, but limited and a single point of failure. Horizontal scaling adds more machines behind a load balancer, which has no hard ceiling and survives the loss of one machine, but it requires the application tier to be stateless.
Stateless means any server can handle any request. Move session data to a shared store such as Redis or into signed tokens, keep uploaded files in object storage, and avoid in-memory state that one request writes and another reads. Then autoscaling can add and remove servers based on CPU or queue length.
Practice. Take a small app that stores sessions in memory, move sessions to Redis, and run two copies behind a proxy.
Load Balancers Core
A load balancer spreads incoming requests across a pool of servers, runs health checks and stops sending traffic to servers that fail them. It also gives clients one stable address while servers come and go. Run at least two balancers, in active-passive or active-active mode, so the balancer is not itself a single point of failure.
Layer 4 Load Balancing works on TCP or UDP connections and only sees IP addresses and ports, so it is very fast and protocol agnostic. Layer 7 Load Balancing reads HTTP: it can route by path or header, terminate TLS, add headers and retry. It costs more CPU but enables smarter routing.
Practice. Configure Nginx or HAProxy to balance two local servers and watch it drop one when you stop it.
Load Balancing Algorithms Core
Common ways to pick a server:
- Round robin: take turns; fine when servers and requests are similar.
- Weighted round robin: bigger servers get a larger share.
- Least connections: prefer whichever server is currently handling the smallest number of open connections; good for long or uneven requests.
- Least response time: favour the fastest recent responder.
- IP or key hash: the same client or key always reaches the same server, useful for caches. Consistent hashing keeps most keys in place when servers are added or removed.
Sticky sessions pin a user to one server; they help with legacy state but break even distribution.
Practice. Explain which algorithm you would choose for WebSocket servers and why.
LB vs Reverse Proxy Recommended
A reverse proxy sits in front of one or more servers and handles requests on their behalf: TLS termination, compression, caching, request limits and hiding internal addresses. A load balancer is a reverse proxy whose main job is spreading load across many servers.
With a single backend, a reverse proxy alone still adds value. Nginx, HAProxy and Envoy do both jobs. Cloud load balancers and API gateways are managed versions. The downside is one more hop and one more component to configure, monitor and make redundant.
Practice. Put Nginx in front of a single app server and use it for TLS, gzip and caching of static files.
Application Layer and Microservices Core
Separating the web tier from the application layer lets you scale each independently: many thin web servers, and worker services sized for their own load. Taking this further, microservices split a system into small services that each own one business capability and its data, deploy independently and talk over the network.
Microservices help large teams move independently but add network failures, distributed transactions, versioned APIs and much harder debugging. A well-structured monolith is the right start for most new products; extract services when a team or a scaling need clearly justifies it. Interviewers like hearing that trade-off.
Practice. Split an e-commerce app into services on paper and list which data each owns and which calls cross the network.
Service Discovery Recommended
When services scale up and down, their addresses change. Service discovery keeps a live registry of healthy instances so callers can find them. With client-side discovery, the caller queries the registry (such as Consul or etcd) and picks an instance. With server-side discovery, the caller hits a load balancer or DNS name that does the lookup.
In Kubernetes, a Service gives a stable DNS name and virtual IP over changing pods, so most teams get discovery for free. Registries rely on health checks and heartbeats to remove dead instances.
Practice. Deploy two versions of a service in Kubernetes and show that callers reach them by Service name only.
Content Delivery Networks Core
A CDN is a worldwide network of edge servers that keep copies of your content close to users, cutting latency and taking load off your origin. It is the standard home for static content hosting: images, scripts, styles, video segments and downloads, often stored in object storage such as S3 behind the CDN.
Pull CDNs fetch a file from your origin the first time a user asks for it, then cache it until its TTL ends; easy to run and good for busy sites. Push CDNs require you to upload content ahead of time; better for large files that change rarely. Version file names (app.3f9c.js) so you can cache for a long time and still release changes instantly.
Practice. Serve a static site from object storage behind a CDN and confirm cache hits in the response headers.
Stage 4 Databases and Storage
Goal. Pick the right kind of database and scale it with indexes, replication, federation and sharding.
SQL vs NoSQL Core
Relational databases such as PostgreSQL and MySQL store rows in tables with a fixed schema, support joins and give ACID transactions. They are the safe default whenever data is related and correctness matters. NoSQL is a loose label for stores that drop some of those features to gain flexible schemas, simple horizontal scaling or a data model tuned for one access pattern.
Choose by access pattern, not fashion. Use SQL for orders, payments, users and anything queried in many ways. Consider NoSQL for huge write volumes, simple key lookups, flexible documents or graph traversals. Many systems use both. The SQL Roadmap roadmap covers relational design.
Practice. For a food delivery app, list each entity and decide whether it belongs in a relational or a NoSQL store.
NoSQL Families Core
- Key-Value Store (Redis, DynamoDB): get and put by key, extremely fast; use for sessions, carts and counters.
- Document Store (MongoDB): JSON-like documents with nested fields and secondary indexes; use for content and catalogues with varied shapes.
- Wide Column Store (Cassandra, HBase): rows keyed by a partition key with many sorted columns, built for very high write rates; use for time series, messages and event logs.
- Graph Databases (Neo4j): nodes and relationships as first-class data; use for social graphs, recommendations and fraud rings where queries hop many links.
Each family is designed around known queries. Design the keys from the questions you will ask, because ad hoc queries are weak or missing.
Practice. Model a chat app's messages in a wide column store with a partition key that keeps each conversation together.
Replication Core
Replication keeps copies of data on several machines for availability and read scaling. In leader-follower replication, one leader takes writes and streams changes to followers, which serve reads. If the leader fails, a follower is promoted. Asynchronous replication is fast but followers lag, so a user may not see their own write; read from the leader just after a write to avoid that.
Multi-leader replication accepts writes in several places (for example, one leader per region) but needs conflict resolution. Leaderless designs such as Cassandra write to several replicas and use read and write quorums: with three replicas, writing to two and reading from two always overlaps by one.
Practice. Set up PostgreSQL streaming replication with one follower and measure the replication lag under load.
Federation and Sharding Core
Federation splits databases by function: users in one, orders in another, products in a third. Each is smaller and handles less traffic, but joins across them move into application code.
Sharding splits one large data set by a key across many databases, each holding a subset of rows. Hash-based sharding spreads load evenly; range-based sharding keeps related keys together but risks hot spots. Pick a key that is in most queries and spreads writes evenly, such as user ID. Cross-shard queries, transactions and rebalancing become hard, so shard only when a single well-tuned primary with replicas is no longer enough. Consistent hashing reduces data movement when shards are added.
Practice. Choose a shard key for a multi-tenant SaaS product and describe what happens when one tenant grows huge.
Denormalization and SQL Tuning Core
Normalised schemas avoid duplicated data, but reads may need many joins. Denormalization copies data where it is read, such as storing an author's name on each post, trading extra writes and the risk of inconsistency for faster reads. It is common once data is sharded, because cross-shard joins are expensive.
Before denormalising, tune: add indexes that match your filters and sort order, read query plans with EXPLAIN, avoid SELECT *, fetch in pages, use connection pooling and keep hot rows small. Most slow databases are missing an index rather than needing a new architecture.
Practice. Take a slow query, read its plan, add one index and compare the before and after timings.
Index Table and Materialized View Recommended
Many NoSQL stores only look up by the primary key. The index table pattern builds a second table keyed by another attribute (for example, email to user ID) so you can query that way too, at the cost of keeping both in sync.
A materialized view stores the precomputed result of an expensive query, such as daily sales per store, and refreshes it on a schedule or when source data changes. Reads become trivial; the trade-off is staleness and the work of refreshing. Both are data management patterns for read-heavy systems.
Practice. Create a materialized view of order totals per customer in PostgreSQL and schedule its refresh.
Stage 5 Caching
Goal. Place caches at the right layers and choose a read and write strategy with a clear invalidation story.
Caching Layers Core
Caching keeps a copy of data somewhere faster than its source. It can happen at every layer:
- Client Caching: the browser or app reuses responses under
Cache-Controlrules. - CDN Caching: edge servers hold static files and some API responses.
- Web Server Caching: a reverse proxy such as Nginx or Varnish stores whole responses.
- Database Caching: the database keeps hot pages in memory; tune its buffer size.
- Application Caching: an in-memory store such as Redis or Memcached holds objects the app computes or loads.
Cache at the highest layer that is still correct. Caching rendered objects avoids repeating work across several queries.
Practice. For one page of a site you know, list what could be cached at each of the five layers.
Cache-Aside Core
The most common pattern. The application checks the cache first; on a miss it reads the database, stores the result in the cache with a TTL and returns it. Only requested data is cached, and a cache failure just means slower reads.
def get_user(uid):
user = cache.get(f"user:{uid}")
if user is None:
user = db.load_user(uid)
cache.set(f"user:{uid}", user, ttl=300)
return userDownsides: each miss costs three trips, and data can be stale until the TTL ends. On update, delete the cache key after writing the database, rather than updating it, to reduce races.
Practice. Add cache-aside with Redis to an endpoint and measure hit rate and latency before and after.
Write-through, Write-behind and Refresh Ahead Core
- Write-through: every write goes to the cache and then synchronously to the database. The cache is always fresh, but writes are slower and the cache fills with data nobody reads.
- Write-behind (write-back): writes go to the cache and are flushed to the database later in batches. Very fast writes, but data can be lost if the cache dies before flushing.
- Refresh Ahead: the cache reloads popular keys shortly before they expire, so readers rarely see a miss. It wastes work if it guesses wrongly which keys will be read.
Most systems use cache-aside for reads, sometimes with write-through for data that must be fresh. Pick write-behind only when losing a few seconds of writes is acceptable, such as view counters.
Practice. For a leaderboard, a user profile and a product price, pick a write strategy for each and justify it.
Eviction, Invalidation and Cache Problems Core
Caches are finite, so they evict entries: LRU removes the least recently used, LFU the least frequently used, and TTLs expire entries by age. Invalidation, making sure stale data is removed when the source changes, is the hard part. Use short TTLs as a safety net even when you delete keys on write.
Know three failure modes. A stampede happens when a hot key expires and thousands of requests hit the database at once; fix it with a lock so one request refills, or with early refresh. Penetration is repeated lookups for keys that do not exist; cache the empty result briefly. Avalanche is many keys expiring together; add random jitter to TTLs.
Practice. Simulate a cache stampede with a load tool, then fix it with a per-key lock and compare database load.
Stage 6 Asynchronism and Messaging
Goal. Move slow work out of the request path with queues, and connect services with events safely.
Asynchronism and Message Queues Core
Asynchronism means accepting a request quickly and doing the slow part later. A message queue (RabbitMQ, Amazon SQS, Kafka) stores messages from producers until consumers process them. This decouples services, smooths traffic spikes and lets a consumer fail without losing work.
Learn the delivery guarantees. At-most-once can lose messages; at-least-once may deliver duplicates, so consumers must be idempotent; exactly-once is only achievable within limited scopes. Failed messages go to a dead-letter queue after a few attempts so one bad message does not block the rest.
Practice. Build a producer and consumer with a queue, make the consumer crash halfway and confirm no message is lost.
Task Queues and Background Jobs Core
A task queue runs units of work such as sending email, resizing images or generating reports on worker processes (Celery, Sidekiq, BullMQ). Background jobs are triggered in two ways: event-driven, when something happens such as an upload, or schedule driven, on a timer such as a nightly clean-up.
Returning results needs a plan, because the caller has moved on: store status in a database for polling, notify via webhook or WebSocket, or publish a completion event. Give every job a timeout, retries with backoff and an idempotent design, and expose queue depth as a metric for autoscaling workers.
Practice. Move image thumbnail generation into a background job and show the job status on the page.
Back Pressure and Queue-Based Load Leveling Core
Queue-Based Load Leveling places a queue between a bursty producer and a service with fixed capacity. The queue absorbs the spike and the service works through it at its own steady rate, instead of falling over.
Queues cannot grow forever. Back pressure is how a system signals "slow down" when it is full: bounded queues that reject new work, HTTP 429 or 503 with a Retry-After header, or reactive streams where consumers request only what they can handle. Without it, memory fills, latency climbs and the whole system fails together.
Practice. Put a bounded queue in front of a slow worker and return 503 when it is full; observe behaviour under load.
Publisher/Subscriber and Competing Consumers Core
In publisher/subscriber messaging, a producer publishes an event to a topic and every subscribed service gets its own copy. The publisher does not know who listens, so new features can subscribe without changing it. Kafka topics, SNS and Google Pub/Sub work this way.
Competing Consumers is the opposite shape: many workers read from one queue and each message goes to exactly one of them, which spreads load and allows scaling by adding workers. Most systems combine both: an "order placed" event fans out to billing, email and analytics, and each of those has its own pool of competing workers.
Practice. Publish an "order placed" event that two services consume, one of them with three competing workers.
Priority Queue, Sequential Convoy and Claim Check Recommended
- Priority Queue: send urgent messages to a separate queue with more workers, so paid customers or password resets are not stuck behind bulk work.
- Sequential Convoy: some messages must be handled in order, such as events for one bank account. Partition by that key so each partition is processed in order while different keys run in parallel; Kafka partitions do this.
- Claim Check: brokers limit message size. Store a large payload in object storage and send only a reference that the consumer uses to fetch it.
These patterns all answer one question: what does the queue need to guarantee, and what can stay outside it?
Practice. Design a queue layout for a print service with urgent and bulk jobs and documents up to 200 MB.
Pipes and Filters, Async Request Reply and Scheduling Agent Supervisor Optional
- Pipes and Filters: break a processing task into independent steps joined by queues, such as upload, scan, transcode and thumbnail; each step scales and fails on its own.
- Async Request Reply: the API answers
202 Acceptedwith a status URL, and the client polls it (or gets a callback) until the result is ready. - Scheduling Agent Supervisor: a scheduler runs the steps of a workflow, an agent calls each remote service, and a supervisor watches for stuck or failed steps and retries or compensates. Workflow engines such as Temporal or AWS Step Functions implement this.
Reach for these when a single request turns into minutes of work across services.
Practice. Design a video upload API with Async Request Reply and a pipes-and-filters back end.
Event-Driven Architecture, Choreography and Orchestration Recommended
In an event-driven system, services announce facts ("payment captured") and others react. Choreography lets each service listen and act on its own, with no central controller: loose coupling, but the overall flow is hard to see and debug. Orchestration puts one coordinator in charge of calling each step, which is easier to follow and change but creates a central dependency.
Use choreography for simple fan-out and orchestration for long business workflows with many steps and failure paths. Either way, publish events reliably with the transactional outbox pattern: write the event to an outbox table in the same transaction as the data change, and let a relay publish it.
Practice. Draw an order flow first as choreography and then as orchestration, and list what each makes harder.
Event Sourcing and CQRS Optional
Event Sourcing stores every change as an immutable event (deposited 50, withdrew 20) instead of only the current state. Current state is rebuilt by replaying events, which gives a complete audit trail and the ability to rebuild new views from history. The costs: replay time (mitigated with snapshots), evolving event formats and a steep learning curve.
CQRS (command query responsibility segregation) separates the write model from one or more read models, each shaped for its queries and updated from events. They pair naturally but are independent. Both add real complexity; use them for domains such as ledgers or audit-heavy systems, not for ordinary CRUD.
Practice. Model a bank account with event sourcing and build a read model that shows the current balance.
Stage 7 Consistency, Availability and Distributed Data
Goal. Choose a consistency model per feature, design for failover and keep multi-service data correct.
Consistency Patterns Core
Three levels of read guarantee after a write:
- Weak Consistency: a read may or may not see a recent write, with no promise. Acceptable for live video or game positions, where newer data replaces old quickly.
- Eventual Consistency: if writes stop, all replicas converge to the same value after a short delay. DNS, email and most NoSQL defaults work this way; good for likes, feeds and search indexes.
- Strong Consistency: every read sees the latest committed write, as if there were one copy. Needed for balances, stock and unique usernames; costs latency and availability during partitions.
In between sit useful guarantees such as read-your-writes and monotonic reads, which fix the most visible anomalies cheaply.
Practice. For a social app, assign a consistency level to posting, liking, following and changing a password.
Availability Patterns and Availability in Numbers Core
Availability is the share of time a system works, often written as nines. 99.9% allows about 8.8 hours of downtime a year, 99.99% about 53 minutes, 99.999% about 5 minutes. Components in series multiply: two services each at 99.9% that are both required give about 99.8%. Components in parallel improve it: two redundant copies at 99.9% give about 99.9999% if failures are independent.
High availability comes from removing single points of failure. Fail-Over is the main pattern: active-passive keeps a standby that takes over when heartbeats stop; active-active runs several copies that all serve traffic. Add replication for data and spread copies across availability zones or regions.
Practice. Compute the availability of a request that needs a load balancer, two app servers in parallel and one database.
Leader Election and Consensus Recommended
Some jobs must have exactly one owner at a time, such as the primary database, the node that runs scheduled tasks or the partition leader in Kafka. Leader election picks that owner and replaces it when it fails. Consensus algorithms such as Raft and Paxos let a majority of nodes agree on the leader and on an ordered log, which is how etcd, ZooKeeper and Consul work.
Use these through existing tools rather than writing your own. Guard against split brain, where two nodes both think they lead, with leases that expire and fencing tokens that storage checks before accepting writes.
Practice. Use a lock with a lease in Redis or etcd to make sure only one worker runs a nightly job.
Distributed Transactions and Compensating Transaction Core
When one business action changes data in several services, a single database transaction is not available. Two-phase commit can coordinate it but blocks if the coordinator fails and is rarely used across services. The saga pattern instead runs a sequence of local transactions, each publishing an event that triggers the next.
If a later step fails, earlier steps are undone by compensating transactions: refund the payment, release the reserved seat. Compensations are new actions, not rollbacks, so design them to be idempotent and to handle cases such as an email that has already been sent.
Practice. Design a travel booking saga for flight, hotel and car with a compensation for each step.
Stage 8 Reliability Patterns and Cloud Design Patterns
Goal. Apply the resiliency and cloud design patterns that keep distributed systems healthy under failure.
Retry and Retry Storm Core
Many failures are brief: a dropped connection or a restarting instance. The Retry pattern tries again after a delay. Use exponential backoff with random jitter, a small maximum number of attempts, and retry only idempotent or idempotency-keyed operations and only on transient errors such as timeouts or 503.
for attempt in range(5):
try:
return call()
except TransientError:
time.sleep(min(10, 0.1 * 2 ** attempt) * random.random())
raise ServiceUnavailable()A retry storm happens when every layer retries: three layers of three attempts turn one failure into 27 calls, flattening a struggling service. Retry at one layer only and use a retry budget.
Practice. Add backoff with jitter to an HTTP client and show the retry spread in logs.
Circuit Breaker and Bulkhead Core
A Circuit Breaker wraps calls to a dependency. When failures pass a threshold it opens and fails fast for a while instead of waiting on timeouts, giving the dependency room to recover. After a cool-down it lets a few test calls through (half-open) and closes again if they succeed. Pair it with a fallback such as cached data.
A Bulkhead isolates resources so one failing part cannot sink the rest, like watertight compartments in a ship. Give each downstream dependency its own connection pool or thread pool; if the recommendations service hangs, it exhausts only its own pool and checkout keeps working.
Practice. Wrap a flaky dependency in a circuit breaker library and show requests failing fast while it is open.
Throttling and Rate Limiting Core
Throttling limits how much work a client or tenant can trigger, protecting the system and keeping usage fair. Common algorithms: token bucket (tokens refill at a steady rate and each request spends one, allowing short bursts), leaky bucket (requests drain at a fixed rate), fixed window counters and sliding window logs or counters.
In a distributed system, keep counters in a shared store such as Redis with atomic increments, and return HTTP 429 with a Retry-After header.
Practice. Implement a token bucket rate limiter backed by Redis and test it with concurrent clients.
Health Endpoint Monitoring Core
Expose endpoints that report whether an instance is alive and ready. A liveness check answers "is the process running and not stuck"; a readiness check answers "can I take traffic now", which includes warm-up and critical dependencies. Load balancers and orchestrators such as Kubernetes call them to route traffic and restart bad instances.
Keep liveness checks cheap and independent of other services, or a database blip will restart every instance at once.
Practice. Add separate liveness and readiness endpoints to a service and wire them into a Kubernetes deployment.
Deployment Stamps, Geodes and Noisy Neighbor Optional
Deployment Stamps run several identical, independent copies of the whole stack, each serving a group of tenants or a region. A failure or bad release affects one stamp, and capacity grows by adding stamps. Geodes go further: each regional node can serve any request, with data replicated between them, so users always hit the nearest healthy region.
Noisy Neighbor is the problem these help with: one tenant's heavy usage slows everyone sharing the same resources. Fix it with per-tenant quotas, throttling, separate pools for large tenants or moving them to their own stamp.
Practice. Design a multi-tenant SaaS layout with stamps and decide when a tenant moves to a dedicated stamp.
Gateway Routing, Gateway Aggregation and Gateway Offloading Core
An API gateway is the single entry point in front of many services. The three gateway patterns describe its jobs. Gateway Routing sends each request to the right service by path or header, so clients see one address. Gateway Aggregation combines several back-end calls into one response, cutting round trips for mobile clients. Gateway Offloading moves shared concerns such as TLS, authentication, rate limiting and logging out of every service into the gateway.
Backends for Frontend takes this further with a separate gateway per client type (web, mobile, partner), each shaped for its screens and owned by that client's team. Keep business logic out of gateways or they become a new monolith.
Practice. Design a gateway for web and mobile clients and list which concerns are offloaded and which calls are aggregated.
Sidecar, Ambassador and Anti-Corruption Layer Recommended
The Sidecar pattern runs a helper process next to each service instance, sharing its lifecycle, to provide logging, configuration or proxying without changing the service's code. Service meshes such as Istio and Linkerd place an Envoy sidecar beside every pod. An Ambassador is a sidecar specialised for outbound calls: it handles retries, circuit breaking and monitoring on the service's behalf, useful for legacy apps you cannot change.
An Anti-Corruption Layer translates between a new system and a legacy one so the old model's quirks do not leak into the new design. It is essential during migrations. These three are design and implementation patterns: they shape how services are built and connected.
Practice. Sketch how a sidecar proxy adds mutual TLS between two services without code changes.
Strangler Fig and Compute Resource Consolidation Recommended
The Strangler Fig pattern replaces a legacy system gradually. Put a facade or proxy in front of it, build new features as new services, and route more and more paths to them until the old system handles nothing and can be switched off. It avoids risky big-bang rewrites.
Compute Resource Consolidation goes the other way for small workloads: pack several light tasks into one compute unit, such as one container host or one function app, to cut cost and overhead. Balance it against isolation: tasks sharing a host can affect each other.
Practice. Plan a strangler migration of a monolith's checkout module in three routing steps.
Security: Gatekeeper, Valet Key and Federated Identity Core
Security patterns limit what an attacker can reach. A Gatekeeper is a hardened front service that validates and sanitises requests before passing them to trusted internal services, which never face the internet directly. The Valet Key pattern gives a client a short-lived, narrowly scoped token, such as a pre-signed upload URL, so it can use a storage resource directly without your servers handling the data.
The Federated Identity pattern delegates sign-in to an identity provider through standards such as OpenID Connect and SAML, so applications receive signed tokens instead of storing passwords. Add TLS everywhere, least-privilege service accounts, secrets in a vault and an External Configuration Store so settings and secrets live outside the deployment package.
Practice. Design login for a product with a web app, a mobile app and an API using an external identity provider.
Stage 9 Performance Antipatterns and Monitoring
Goal. Recognise the common performance antipatterns and set up monitoring that tells you what is wrong.
Performance Antipatterns: Data Access Recommended
- Busy Database: business logic pushed into stored procedures or heavy queries overloads the shared database; move processing to the application tier.
- Chatty I/O: many tiny calls, such as one query per item in a loop (the N+1 problem); batch or join instead.
- Extraneous Fetching: loading whole rows or tables when only a few fields are needed; select columns and filter in the database.
- Monolithic Persistence: one database for every kind of data and workload; split by usage, such as a separate store for logs or analytics.
- No Caching: repeatedly fetching data that rarely changes; add a cache with a sensible TTL.
Finding these needs measurement, not guesswork: profile, trace and read query plans.
Practice. Find an N+1 query in an ORM-based app and fix it with eager loading or a batch query.
Performance Antipatterns: Compute and Traffic Recommended
- Busy Frontend: heavy work done in the web tier slows user requests; move it to background workers.
- Synchronous I/O: threads block waiting on network or disk, so capacity collapses under load; use asynchronous calls.
- Improper Instantiation: creating expensive objects such as HTTP clients or connection pools per request instead of reusing one shared instance.
- Retry Storm: uncoordinated retries amplify an outage; use backoff, jitter and budgets.
- Noisy Neighbor: one tenant starves others on shared resources.
Each has a recognisable symptom in metrics: high CPU on the web tier, thread pool exhaustion, rising connection counts or a sudden traffic multiplier during failures.
Practice. Create an HTTP client per request in a small service, load test it, then reuse one client and compare.
Monitoring and Instrumentation Core
You cannot operate what you cannot see. Instrumentation adds the signals: metrics (counts, rates, latencies), structured logs and distributed traces that follow one request across services. OpenTelemetry is the common standard for collecting them; Prometheus, Grafana and hosted tools store and show them.
Monitor at several levels. Health Monitoring and Availability Monitoring check that services are up and reachable. Performance Monitoring tracks latency percentiles, throughput, errors and saturation. Security Monitoring watches for failed logins, unusual access and configuration changes. Usage Monitoring shows how features are used and drives capacity planning and billing.
Practice. Instrument a service with OpenTelemetry and trace one request across two services.
Visualization & Alerts Core
Dashboards turn metrics into a picture of system health. Start each with the four golden signals: latency, traffic, errors and saturation. Define service level objectives such as "99.9% of requests under 300 ms over 30 days", and alert on burning through the error budget rather than on every CPU spike.
Good alerts are actionable, rare and routed to the right on-call person with a runbook. Alert on user-facing symptoms; use cause-based metrics for diagnosis. Review alerts after incidents and delete ones nobody acts on. The Kubernetes Roadmap and AWS Roadmap roadmaps cover platform-specific tooling.
Practice. Build a dashboard with the four golden signals for one service and write one SLO-based alert.
Stage 10 Classic Designs and Interview Practice
Goal. Design the common interview systems end to end within 45 minutes and defend each trade-off.
URL Shortener Core
Requirements: create a short code for a long URL, redirect fast, maybe custom aliases and expiry. It is read heavy, often 100 reads per write. Generate codes with a counter encoded in base 62 (seven characters give over three trillion codes) using ranges handed out to each server, or hash the URL and handle collisions.
Store code to URL in a key-value store or a sharded relational table. Serve redirects from a cache, since a few links get most clicks. Use 301 if you want browsers to cache the redirect, or 302 if you need every click counted. Send click events to a queue for analytics rather than writing them in the request path.
Practice. Design a URL shortener for 100 million new links a month in system design practice, with estimates.
Rate Limiter Core
Requirements: limit requests per user or API key, work across many servers, add little latency, and return clear errors. Place it in the API gateway or as middleware. Token bucket is a good default: it allows short bursts while enforcing an average rate.
Keep counters in Redis with atomic operations or a short Lua script so check-and-decrement is one step. Decide whether to fail open (allow traffic if Redis is down, favouring availability) or fail closed (block, favouring protection). Return HTTP 429 with remaining quota headers. For very high volume, keep local counters and sync to the shared store periodically, accepting slight overshoot.
Practice. Design a distributed rate limiter in system design practice and explain its behaviour when Redis is unavailable.
News Feed Core
Requirements: users post, follow others and see a ranked feed of followed accounts. Two strategies. Fan-out on write pushes each new post ID into every follower's feed list in a cache at posting time, giving fast reads but expensive writes for accounts with millions of followers. Fan-out on read assembles the feed when requested, which is cheap to write but slow to read.
The usual answer is hybrid: fan-out on write for normal users, and merge posts from celebrity accounts at read time. Store posts in a sharded store, feed lists in Redis, media in object storage behind a CDN, and rank with a separate service.
Practice. Design a news feed in system design practice for 200 million daily users and handle a celebrity with 50 million followers.
Chat System Core
Requirements: one-to-one and group messages, online presence, delivery receipts and history. Clients hold WebSocket connections to chat servers; a session service records which server each user is connected to. A message goes to the sender's server, is stored, then routed to the recipient's server or queued for push notification if they are offline.
Store messages in a wide column store partitioned by conversation ID and ordered by a time-sortable message ID. Presence uses heartbeats with a timeout. Group messages fan out to members; very large groups switch to pull. Keep per-conversation order with sequence numbers and let clients deduplicate on retry.
Practice. Design a chat system in system design practice, focusing on message ordering and offline delivery.
More Classic Designs Recommended
Rehearse these next; each stresses a different building block:
- Typeahead search: trie or prefix index with precomputed top results, cached at the edge.
- File storage and sync: chunked uploads, content hashes for deduplication, metadata database, change notifications.
- Video streaming: transcoding pipeline, adaptive bitrate segments, CDN delivery.
- Ride matching: geospatial index such as geohash cells, frequent location updates, matching service.
- Web crawler: URL frontier queue, politeness per domain, deduplication with hashes and Bloom filters.
- Notification service: queues per channel, templates, retries, user preferences and rate limits.
- Distributed key-value store: consistent hashing, replication, quorums, hinted handoff.
Practice. Pick two of these and design each in system design practice within 40 minutes.
Practising Design Interviews Core
Design skill grows from repetition with feedback. Use system design practice to sketch architectures against a timer, then compare your design with the requirements: did you cover scale, failure and data growth? Practise explaining trade-offs aloud, since interviewers grade reasoning more than diagrams.
Build a short personal checklist: requirements, estimates, API, data model, high-level diagram, one deep dive, bottlenecks, failure handling, monitoring. Do mock interviews with a peer and swap roles; acting as interviewer shows you what good answers sound like.
Practice. Do one timed mock design per week for six weeks and keep notes on what the interviewer probed.
Projects to build
Build these in order. Each one should end up in a public repository with a short README that explains your decisions.
- Scalable URL shortenerBeginner
Build a URL shortener with an API, a relational database and Redis cache-aside for redirects. Run two app instances behind Nginx and load test it, then record p50 and p99 latency with and without the cache.
Load Balancers · Cache-Aside · Estimation
- Distributed rate limiterIntermediate
Implement token bucket rate limiting as middleware backed by Redis, shared by three app instances. Return 429 with retry headers, test it under concurrent load, and decide and document its behaviour when Redis is unreachable.
Throttling · Redis · Failure modes
- Image processing pipelineIntermediate
Let users upload images directly to object storage with pre-signed URLs. Trigger a queue message per upload, process thumbnails with competing workers, and expose job status through an Async Request Reply endpoint. Add a dead-letter queue and retries with backoff.
Valet Key · Message Queues · Competing Consumers · Background Jobs
- Order system with a sagaAdvanced
Build order, payment and inventory services that communicate through events. Implement an orchestrated saga with compensating transactions, the outbox pattern for reliable publishing, idempotent consumers and a circuit breaker on the payment call.
Microservices · Compensating Transaction · Idempotent Operations · Circuit Breaker
- Observable chat serviceAdvanced
Build a WebSocket chat service with several connection servers, a session registry in Redis and messages stored in a wide column or partitioned table. Add OpenTelemetry tracing, a golden signals dashboard and one SLO alert, then kill a server and watch clients reconnect.
WebSockets · Sharding · Monitoring · Fail-Over
Interview questions
Try to answer each question out loud before you open the answer.
What is the difference between horizontal and vertical scaling?Junior
Vertical scaling moves to a bigger machine with more CPU and memory. It needs no code changes but has a ceiling, gets expensive, and leaves a single point of failure. Horizontal scaling adds more machines behind a load balancer. It scales much further and survives individual failures, but the application must be stateless, and data stores need replication or sharding. In practice I scale the stateless tier horizontally early and scale databases vertically as long as it is reasonable.
Explain the CAP theorem with an example.Mid
When a network partition splits a distributed store, each side must either keep serving requests with possibly stale or conflicting data, or reject requests until the partition heals. You cannot have both full availability and strong consistency during a partition. A shopping cart can stay available and merge later; an account balance should reject writes on the minority side. PACELC adds that without a partition you still trade latency against consistency.
How would you design caching for a read-heavy API?Mid
I would use cache-aside with Redis: check the cache, load from the database on a miss and set a TTL. On writes I update the database, then delete the key. I would add a CDN for public responses and HTTP cache headers for clients. To protect the database I would use request coalescing or a lock for hot keys, jittered TTLs to avoid mass expiry, and short caching of not-found results. I would track hit rate and latency.
Layer 4 vs Layer 7 load balancer: when would you use each?Mid
Layer 4 balances TCP or UDP connections by address and port without reading content. It is very fast, works for any protocol and suits databases, raw TCP services or the outer tier in front of other balancers. Layer 7 understands HTTP, so it can route by path or host, terminate TLS, add headers, retry and do canary splits. I use Layer 7 for web and API traffic and Layer 4 when I need raw throughput or a non-HTTP protocol.
How do you choose a shard key?Senior
The key should appear in most queries so they hit one shard, spread data and writes evenly, and not create hot spots. User ID or tenant ID is common. Time alone is bad because all new writes land on one shard. I check the biggest entities: one huge tenant can still overload a shard, so I may split large tenants or add a suffix. I also plan rebalancing, typically with consistent hashing or many small logical shards mapped to fewer physical ones.
What happens when a message is delivered twice, and how do you handle it?Mid
Most queues guarantee at-least-once delivery, so a consumer may crash after acting but before acknowledging, and the message is redelivered. I make consumers idempotent: record processed message IDs in the same transaction as the side effect, use upserts or conditional updates, or pass idempotency keys to downstream APIs. For ordering, I partition by entity key so each entity's events are processed in sequence.
Explain the circuit breaker pattern.Mid
It wraps calls to a remote dependency and tracks failures. While closed, calls pass through. When the failure rate crosses a threshold, it opens and fails immediately for a cool-down period, which stops threads piling up on timeouts and gives the dependency time to recover. Then it moves to half-open and lets a few trial calls through; success closes it, failure opens it again. I pair it with a fallback and with timeouts and limited retries.
Design a URL shortener. Where are the bottlenecks?Mid
Writes are few, reads are many. I generate codes from a counter encoded in base 62, giving each app server a block of IDs so there is no central bottleneck. I store mappings in a key-value store and cache popular codes in Redis, behind a load balancer and optionally a CDN. Bottlenecks are the ID generator, which ranges solve, and hot links, which the cache solves. Analytics go to a queue so redirects stay fast.
How would you design a news feed for a large social network?Senior
Posts are stored in a sharded store and media in object storage behind a CDN. For feeds I use fan-out on write: when a user posts, workers push the post ID into each follower's cached feed list. For accounts with millions of followers that is too costly, so their posts are merged at read time. A ranking service orders candidates. Feed lists are capped in length, and older items are paged from storage.
How do you keep data consistent across microservices without distributed transactions?Senior
I use sagas: each service runs a local transaction and publishes an event that triggers the next step, either through choreography or an orchestrator. If a step fails, earlier steps run compensating actions such as refunds. To publish reliably, I write events to an outbox table in the same transaction as the data and relay them to the broker. Consumers are idempotent, and the UI accepts that some states are pending for a short time.
How would you design a rate limiter for a public API?Mid
I would put it in the gateway, keyed by API key, using a token bucket so short bursts are allowed. Counters live in Redis, updated atomically with a Lua script. Requests over the limit get 429 with a Retry-After header and quota headers. I decide on failing open if Redis is down to keep the API usable, with local fallback limits. For different plans, I store limits per key in configuration.
How do you make a system highly available?Mid
Remove every single point of failure: at least two instances of each service across availability zones, redundant load balancers, database replicas with automatic failover, and multi-region where the business needs it. Add health checks so traffic avoids bad instances, timeouts, retries with backoff, circuit breakers and graceful degradation. Then measure it with SLOs, and test failover regularly, because untested failover often fails when needed.
Common mistakes
- Drawing boxes before agreeing requirements; spend the first five minutes on features, scale and priorities.
- Skipping estimates, then choosing sharding or exotic databases a single server could handle; do the arithmetic first.
- Naming technologies without reasons; for every component, state what problem it solves and what it costs.
- Ignoring failure; for each component say what happens when it dies and how the system recovers.
- Treating microservices as the default; start with a modular monolith unless scale or team size justifies more.
- Adding a cache without an invalidation plan; decide TTLs and write behaviour before you add it.
- Assuming queues deliver exactly once; design idempotent consumers and dead-letter handling.
Ready-for-interview checklist
- I can run the six-step framework for any design prompt within 45 minutes.
- I can estimate requests per second, storage and bandwidth from user numbers.
- I can explain DNS, TCP, UDP and HTTP and how a request reaches a server.
- I can compare REST, gRPC and GraphQL and pick one for a given client.
- I can explain Layer 4 and Layer 7 load balancing and common algorithms.
- I can choose between SQL and NoSQL families and justify it by access pattern.
- I can explain replication, federation and sharding and choose a shard key.
- I can describe cache-aside, write-through, write-behind and refresh-ahead and their risks.
- I can design with queues, pub/sub and competing consumers, including idempotency.
- I can explain CAP, PACELC and the consistency levels with examples.
- I can apply retries with backoff, circuit breakers, bulkheads and throttling.
- I can name the performance antipatterns and how to detect each.
- I have designed a URL shortener, rate limiter, news feed and chat system aloud.
Related roadmaps
- Data Structures and Algorithms RoadmapFrom arrays to graphs and DP, organised by interview pattern.
- Computer Science RoadmapTheory every engineer leans on: complexity, memory, OS, networks, compilers.
- Software Design and Architecture RoadmapClean code, SOLID, design patterns, DDD and architecture styles.
- API Design RoadmapREST, gRPC, GraphQL, versioning, auth, pagination and rate limits.
- Redis RoadmapData types, caching patterns, persistence, clustering and streams.
- Kubernetes RoadmapPods, deployments, services, config, scaling and operations.
FAQ
How long does it take to learn system design for interviews?
With a few years of back-end experience, two to three months of steady study and weekly mock designs is typical. Newer engineers need longer because many building blocks are unfamiliar. The fastest progress comes from designing systems aloud and getting feedback, not from reading alone.
Do junior engineers get system design interviews?
Sometimes, but usually lighter: an object-oriented design, an API design or a simple service. Full distributed design rounds are standard for mid-level and senior roles. Juniors still benefit from the first four stages of this roadmap because they explain how the systems they work on fit together.
Should I learn DSA or system design first?
For most people, data structures and algorithms first, because coding rounds appear at every level. Start system design when you can solve medium problems reliably, or in parallel if you target senior roles. See the Data Structures and Algorithms Roadmap roadmap.
Do I need cloud certifications for system design interviews?
No. Interviews test concepts that apply on any provider: load balancing, caching, storage, queues and failure handling. Naming a managed service is fine, but explain what it does. Hands-on cloud experience helps you speak concretely, and the AWS Roadmap roadmap covers one provider in depth.