Scale a Web App from One Server to 10 Million Users

You are the first engineer at a startup building a recipe-sharing web and mobile app. Users browse recipes, save favourites, upload photos and leave comments. Today everything (web server, application code and database) runs on one rented virtual machine, and it serves a few hundred users.

The app is growing fast, and the founders expect 10 million registered users and about 2 million daily actives within two years. They do not want a rewrite every six months, and they cannot afford a big infrastructure team.

In 45 minutes, walk the architecture forward one stage at a time. For each stage, say what breaks first, what you add, roughly at what scale it becomes necessary, and what it costs in complexity. End with the architecture you would run at 10 million users and how you would know when to take the next step.

Sketch on the HLD board, write your answer in the Design doc, and size it with the Estimator. Start the mock interview from the timer for a 45-minute run.

Design Path · Core Building Blocks · problem 8 of 70

Ask before you design. Try answering each yourself, then compare.

1. What is the read-to-write ratio?
   Typical answer

   Heavily read: people browse far more than they post, roughly 20 reads per write.

2. What does the target scale look like?
   Typical answer

   10 M registered users, 2 M daily actives, about 50 requests per active user per day, with an evening peak around 3x average.

3. How much media?
   Typical answer

   Each new recipe has a few photos averaging 500 KB; about 100k recipes are posted a day at full scale.

4. What availability is expected?
   Typical answer

   99.9% at first, rising to 99.95% at scale. Short maintenance windows become unacceptable once users are global.

5. Is the team big enough to run many services?
   Typical answer

   No. Prefer managed services and a modular monolith; split services only where scaling or team ownership demands it.

6. Do we need multiple regions?
   Typical answer

   One primary region plus disaster recovery in a second region is enough for now. Global users are served static content from the edge.

Requirements

Functional

• Browse and search recipes, view recipe pages with photos and comments.
• Sign up, log in, save favourites, post recipes with photos and comment.
• Send emails such as welcome and weekly digest.

Non-functional

• Page API p95 under 200 ms; images load from a nearby edge.
• Handle ~3,500 requests per second at peak at 2 M DAU, with headroom to 3x.
• 99.95% availability at scale; no single machine whose loss takes the site down.
• No data loss beyond a few seconds on a database failure; daily backups kept 30 days.

Out of scope

• Building search ranking or recommendations in depth.
• Active-active multi-region writes.
• Payments.

Back-of-the-envelope estimates

• Requests: 2 M DAU × 50 requests = 100 M a day ≈ 1,160 per second, ~3,500 at peak. About 5% are writes, ~175 per second at peak.
• App tier: at ~800 dynamic requests per second per server, peak needs 5 servers; run 6-8 for headroom and rolling deploys.
• Database reads: ~3,300 reads/s at peak; with a 90% cache hit rate only ~330 reach the database, which one primary plus a replica handles easily. Without the cache, the database is the first thing to fall over.
• Structured data: 5 M writes a day × ~2 KB ≈ 10 GB a day, ~3.6 TB a year. That single number is what eventually forces sharding or archiving.
• Media: 100k recipes × 3 photos × 500 KB ≈ 150 GB a day, ~55 TB a year, in object storage, never in the database.
• Bandwidth: images dominate egress; a CDN serving 95% of image bytes keeps origin bandwidth small.

Reads and images drive the early stages (cache and CDN); data growth and write volume drive the late ones (replicas, sharding).

API

GET  /v1/recipes?cursor=&tag=           -> {items, nextCursor}
GET  /v1/recipes/{id}                    -> recipe + top comments
POST /v1/recipes  {title, steps, photoKeys[]}   -> 201 {id}
POST /v1/uploads                         -> {uploadUrl, photoKey}  (pre-signed)
POST /v1/recipes/{id}/comments {text}    -> 201
PUT  /v1/me/favourites/{recipeId}        -> 204

Photos are uploaded straight to object storage with a pre-signed URL, so large uploads never pass through (or tie up) the app servers.

Data model

A relational database holds the core data, because users, recipes, comments and favourites are relational and need transactions: users(id, email, ...), recipes(id, author_id, title, body, created_at), comments(id, recipe_id, author_id, text), favourites(user_id, recipe_id). When writes outgrow one primary, shard by user_id for user-owned data and by recipe_id for comments, so the hot queries (a recipe page, a user's favourites) stay on one shard.

Photos live in object storage keyed by photos/{photoKey}, served through the CDN. Sessions and hot recipe pages live in an in-memory cache. Search uses a separate search index fed asynchronously from the database.

High-level design

The final architecture at 10 million users: DNS points the site at a load balancer, while static assets and photos use a CDN hostname. The browser loads images from the CDN, which fetches from object storage only on a miss.

Read path: the load balancer forwards an API request to any of the stateless app servers. Sessions are tokens checked against the shared cache, so any server can handle any user. The server looks for the rendered recipe in the cache, and on a miss reads from a read replica (or the primary if the user just wrote and needs read-your-writes), then fills the cache.

Write path: writes go to the primary of the right shard, which replicates asynchronously to its read replicas and to a standby in the disaster recovery region. The app server invalidates affected cache keys. Slow work (thumbnail generation, emails, search indexing) is pushed to a job queue and handled by background workers, which write thumbnails to object storage.

Metrics, logs and alerts cover every tier, and the app tier autoscales on CPU and latency.

Open the HLD board and press Reference to see the architecture diagram.

Deep dives

Stage 1-2: one box, then split the database (up to ~10k users)

Start with a single server running the app and the database: cheapest, simplest, and fine for a few thousand users. The first problems are that the app and the database fight over memory and CPU, and a single crash or deploy takes everything down. Move the database to its own machine (ideally a managed database with automated backups and a standby). Now each can be sized independently: the database wants memory and fast disks, the app wants CPU. Scaling up a machine is a fine move here; it stops working once the largest affordable instance is not enough or a single machine's failure costs too much.

Stage 3: load balancer and a stateless web tier (~10k-100k users)

Put a load balancer in front of two or more app servers. This only works if servers are stateless: move sessions out of server memory into a shared cache or signed tokens, store uploads in object storage instead of local disk, and keep configuration in the environment. Then any server can serve any request, a crashed server just drops out of the pool, deploys roll one server at a time, and autoscaling adds servers in minutes during the evening peak. Health checks on the load balancer remove unhealthy servers automatically.

Stage 4: cache, CDN and read replicas (~100k-2M users)

The database now becomes the bottleneck, mostly from repeated reads of the same popular recipes. Add a cache with cache-aside reads and delete-on-write invalidation; a 90% hit rate cuts database reads tenfold. Serve images, CSS and JavaScript from a CDN so most bytes never reach your servers. Add one or two read replicas for queries that miss the cache, and send reads that must see the user's own fresh write to the primary. Move slow tasks (thumbnails, emails) to a queue and workers so requests return quickly and spikes are absorbed.

Stage 5: sharding, DR region and observability (~2M-10M users)

Eventually the primary hits its limits: write throughput, or several terabytes that make backups, schema changes and failover slow. Before sharding, archive cold data and split off obvious tables (for example comments) into their own database. When that is not enough, shard by user ID through a lookup or consistent hashing layer, accepting that cross-shard queries and joins move into the application. Replicate to a standby region for disaster recovery with a tested failover runbook. Finally, invest in metrics, logs, tracing and alerts on latency and error rates: at this size, knowing what broke matters as much as the architecture.

Failure modes

• If an app server dies, the load balancer stops routing to it after a failed health check and autoscaling replaces it; no user state is lost because servers are stateless.
• If the cache cluster fails, all reads hit the database; size replicas for a cold-cache period or shed non-critical traffic.
• If the database primary fails, the standby is promoted within a minute; writes fail briefly and the last few seconds of asynchronous replication may be lost.
• If the job queue backs up, user-facing requests are unaffected; thumbnails and emails are delayed and workers scale out.
• If the primary region goes down, fail over to the DR region via DNS, accepting minutes of downtime and a small data-loss window.

Core algorithm in code

import math

# Rough per-unit capacities used for planning (order-of-magnitude, tune with load tests).
APP_RPS_PER_NODE = 800          # dynamic requests per app server
DB_READS_PER_PRIMARY = 15_000   # simple indexed reads/s one well-tuned primary handles
DB_WRITES_PER_PRIMARY = 3_000
CACHE_HIT_RATE = 0.9


def plan(dau, req_per_user=50, write_share=0.05, peak=3):
    rps = dau * req_per_user / 86_400 * peak
    writes, reads = rps * write_share, rps * (1 - write_share)
    db_reads = reads * (1 - CACHE_HIT_RATE)
    steps = []
    app = max(2, math.ceil(rps / APP_RPS_PER_NODE * 1.3))         # 30% headroom, never fewer than 2
    steps.append("%d app servers behind a load balancer" % app if rps > APP_RPS_PER_NODE * 0.5 else "1 server is enough (add a 2nd for availability)")
    if reads > 2_000:
        steps.append("cache in front of DB (DB reads drop to %d/s)" % db_reads)
    replicas = math.ceil(db_reads / DB_READS_PER_PRIMARY)
    if replicas > 1:
        steps.append("%d read replicas" % replicas)
    shards = math.ceil(writes / (DB_WRITES_PER_PRIMARY * 0.6))
    if shards > 1:
        steps.append("shard writes across %d primaries" % shards)
    return rps, steps


for dau in [1_000, 100_000, 2_000_000, 50_000_000]:
    rps, steps = plan(dau)
    print("DAU %-10s peak %8.0f req/s -> %s" % ("{:,}".format(dau), rps, "; ".join(steps)))

Trade-offs

• Scaling up one machine is simple and keeps strong consistency but has a hard ceiling and a single point of failure; scaling out removes both but adds coordination.
• Caching cuts database load dramatically but introduces staleness and invalidation bugs.
• Read replicas add read capacity cheaply but lag behind the primary, breaking read-your-writes unless reads are routed carefully.
• Sharding scales writes and storage but makes cross-shard joins, transactions and rebalancing the application's problem.
• Splitting a monolith into services lets teams scale independently but costs operational overhead a small team may not afford.

What a strong answer shows

• Evolves the system in stages tied to concrete bottlenecks and rough user counts.
• Makes the web tier stateless before scaling it out.
• Uses caching and a CDN as the first defence for a read-heavy workload.
• Understands replica lag and read-your-writes.
• Treats sharding as a late, deliberate step with clear costs.
• Mentions monitoring as the way to know when the next step is needed.

Mistakes to avoid

• Jumping straight to microservices and sharding for a product with a few hundred users.
• Adding servers behind a load balancer while sessions or uploads still live on local disk.
• Storing images in the database.
• Adding read replicas without handling replication lag.
• Never saying what metric triggers each scaling step.

Follow-ups to expect

• How would you serve users in Asia with low latency? (CDN for static content first, then a regional read replica and cache; writes still go to the primary region.)
• How do you run schema migrations without downtime at this size? (Expand-and-contract: add columns, dual-write, backfill, then remove the old path.)
• How do you rebalance shards later? (Many logical shards mapped onto fewer physical databases, moved one logical shard at a time.)
• What would you split out of the monolith first? (Components with distinct scaling needs, such as media processing or search.)

Practice next