Design a Content Delivery Network

Build a content delivery network that thousands of customer websites and apps put in front of their own servers. Their users around the world download images, scripts, software updates and video segments, and expect them to load as if the files were hosted in their own city.

Customers point their domain at you and keep their origin servers small. They expect most bytes to be served without touching the origin, changes to appear within seconds when they purge, and their sites to stay up when one of your locations goes dark.

In 45 minutes, cover how a user is routed to a nearby edge location, how caches inside a location are organised, how cache misses reach the origin without overwhelming it, how invalidation works globally, and how you measure and protect the hit ratio.

Sketch on the HLD board, write your answer in the Design doc, and size it with the Estimator. Start the mock interview from the timer for a 45-minute run.

Design Path · Data & Infrastructure · problem 39 of 70

Ask before you design. Try answering each yourself, then compare.

1. What kind of content?
   Typical answer

   Static and cacheable content: images, scripts, downloads and video segments. Dynamic, per-user pages pass through but are mostly uncached.

2. What scale?
   Typical answer

   About 20B requests a day, ~700k requests/s at peak, averaging ~500 KB per response, served from roughly 200 points of presence (PoPs).

3. How do customers control caching?
   Typical answer

   Through standard cache headers from their origin, plus per-path rules and TTL overrides in our console.

4. How fast must purges take effect?
   Typical answer

   A purge of a URL or tag should reach every PoP within 5 seconds at p99.

5. Do we terminate TLS for customer domains?
   Typical answer

   Yes, at the edge, with certificates we manage or that customers upload.

6. What hit ratio is expected?
   Typical answer

   Above 95% of requests served from the edge for typical static sites, and origin load reduced by at least 99%.

Requirements

Functional

• Route each user to a nearby, healthy PoP for a customer's domain.
• Serve cached content honouring cache headers and per-customer rules; fetch from origin on a miss.
• Purge content by URL, prefix or tag across all PoPs.
• Terminate TLS for customer domains and report usage and hit ratio per customer.

Non-functional

• ~700k requests/s and ~3 Tbps peak egress across ~200 PoPs.
• Cache hit time to first byte under 30 ms for users near a PoP; edge hit ratio above 95%.
• Purge propagation under 5 s p99 worldwide.
• Survive the loss of a whole PoP with traffic rerouted in under a minute; 99.99% availability.

Out of scope

• Running customer code at the edge (follow-up).
• DDoS scrubbing internals and bot detection models.
• Live video encoding and packaging.

Back-of-the-envelope estimates

• Requests: 20B/day ≈ 230k/s average, ~700k/s at peak.
• Egress: 230k/s × 500 KB ≈ 115 GB/s, about 0.9 Tbps average and ~3 Tbps peak. Large metros carry far more than the average PoP, so the busiest sites need hundreds of Gbps.
• Servers: one edge server pushes roughly 40–80 Gbps from memory and SSD with kernel-bypass or sendfile-style I/O, so peak egress needs ~60–100 servers worldwide in theory; redundancy and geographic spread push that to thousands.
• Cache size: ~10M new or changed objects/day × 500 KB ≈ 5 TB/day. A PoP with 30 servers × ~20 TB of SSD holds ~600 TB, enough for the hot working set of most customers.
• Origin load: at a 95% edge hit ratio, 5% of 0.9 Tbps ≈ 45 Gbps would go upstream; a shield tier with its own ~90% hit ratio cuts origin traffic to ~5 Gbps.

Egress bandwidth and hit ratio drive the design: every point of hit ratio saves origin capacity and transit cost.

API

GET  https://cdn.customer.com/img/hero.jpg      (user, via edge)
     -> 200 + Cache-Control, Age, X-Cache: HIT|MISS
POST /v1/zones/{zone}/purge   {urls?[], prefixes?[], tags?[]}  -> {purgeId}
GET  /v1/purges/{purgeId}                                 -> {popsDone, popsTotal}
PUT  /v1/zones/{zone}/rules  [{match:"/static/*", ttl:86400, cacheKey:["path"]}]

Purges are asynchronous with a status endpoint, because a global fan-out can be confirmed but never made instant; customers can wait on it in their deploy scripts.

Data model

Cache entries on each edge server: key = hash of (zone, normalised URL, selected headers such as encoding), value = body plus metadata (expiry, ETag, tags, purge generation). Hot objects sit in RAM, the rest on local SSD, with LRU-style eviction weighted by size and hit frequency.

Zone configuration: customer domains, origins, cache rules, certificates. Stored centrally in a replicated SQL database and pushed to every PoP as versioned snapshots, because the edge must keep working when the control plane is unreachable.

Purge log: an ordered, replicated log of purge commands with sequence numbers, so each PoP can tell whether it has applied everything up to a point. Request logs stream to an analytics store for billing and hit-ratio reports.

High-level design

Finding a PoP. The user's resolver asks our DNS for the customer's hostname. Our DNS answers with addresses chosen by a request mapper that knows each PoP's health, load and measured latency to the resolver's network. Many deployments also announce the same IP addresses from every PoP (anycast), so the internet's routing delivers packets to a nearby site and a dead PoP simply stops announcing.

Inside the PoP. A layer-4 load balancer spreads connections across edge servers, which terminate TLS. The server that receives a request hashes the cache key to find which server in the PoP owns that object, so each object is stored once per PoP rather than on every machine; very popular objects are also kept in every server's memory. On a hit, the object is returned straight away.

On a miss, the owning server asks a regional shield PoP instead of the origin. The shield collapses simultaneous requests for the same object into one origin fetch, caches the response and passes it back, so the origin sees roughly one request per object per region rather than one per edge server.

Control. Customers send purges and configuration to the control plane, which writes them to a replicated bus that every PoP and shield subscribes to. Edge servers ship access logs to an analytics pipeline for usage, billing and hit-ratio dashboards.

Open the HLD board and press Reference to see the architecture diagram.

Deep dives

Routing users: DNS mapping and anycast

DNS-based mapping picks a PoP per resolver network using latency measurements and current load, which allows fine-grained steering but reacts only as fast as DNS TTLs (typically 20–60 seconds) and sees the resolver's location rather than the user's; the client-subnet extension helps. Anycast advertises one prefix from every PoP and lets BGP choose, which fails over almost instantly and absorbs attack traffic locally, but gives less control and can shift a long-lived connection mid-flight when routes change. A common blend uses anycast for DNS and for most traffic, with DNS steering used to drain or rebalance overloaded PoPs.

Making a PoP one big cache

If each of 30 servers caches independently, the PoP stores 30 copies of hot objects and its effective size shrinks to one server's disk. Hashing the cache key onto the PoP's servers, with consistent or rendezvous hashing, makes each object live on one owner, so capacity adds up and adding or removing a server only moves that server's share of keys. The receiving server proxies to the owner over the fast local network. The very hottest objects break this rule: they are replicated into every server's RAM so a single viral file cannot saturate one machine's network card.

Origin shielding and request collapsing

When a new release goes out, every PoP misses at once. Without protection, 200 PoPs times many concurrent users can send thousands of identical requests to a small origin. Route all misses for a customer through one shield PoP per region (or one globally for small origins). At each tier, collapse concurrent misses for the same key: the first request fetches while the rest wait on it. Serving stale content while revalidating in the background, and serving stale on origin errors, keeps users fast and the site up even when the origin is struggling.

Fast, reliable invalidation

Purges are appended to a replicated log with sequence numbers and pushed to every PoP over persistent connections; each PoP acknowledges the highest sequence it has applied, which drives the purge status API. Purging by URL simply deletes the entry. Purging by prefix or tag is done lazily: record a purge generation per tag, and treat any cached object whose stored generation is older as a miss. A PoP that was disconnected replays the log from its last sequence before serving, or flushes the affected zone if it fell too far behind. Versioned URLs remain the best practice for assets, since they need no purge at all.

Failure modes

• If a PoP fails, it stops announcing its anycast routes and the mapper removes it from DNS answers; users shift to the next nearest PoP, whose hit ratio drops briefly while it warms.
• If an edge server dies, the PoP's hash ring drops it and only its share of objects becomes cold, refilled from the shield.
• If a customer's origin is down, serve stale content within a configurable grace period and return clear errors only for objects never cached.
• If the control plane is unreachable, PoPs keep serving with their last configuration snapshot; new purges queue and are replayed by sequence number on reconnect.
• If a shield PoP fails, misses fall back to a secondary shield or directly to origin with stricter rate limits and request collapsing.

Core algorithm in code

import hashlib
from collections import OrderedDict


def pick_server(url, servers):
    """Rendezvous hashing: each URL lives on one server in the PoP; removing a server moves only its URLs."""
    return max(servers, key=lambda s: hashlib.md5(f"{s}|{url}".encode()).hexdigest())


class Origin:
    def __init__(self):
        self.fetches, self.version = 0, {}

    def get(self, url):
        self.fetches += 1
        return f"{url} v{self.version.get(url, 1)}"


class CacheTier:
    """LRU cache with TTL. On a miss, asks the tier behind it (shield or origin)."""

    def __init__(self, name, capacity, upstream, ttl=60):
        self.name, self.cap, self.up, self.ttl = name, capacity, upstream, ttl
        self.items = OrderedDict()                      # url -> (body, expires_at)
        self.hits = self.misses = 0

    def get(self, url, now):
        entry = self.items.get(url)
        if entry and entry[1] > now:
            self.items.move_to_end(url)
            self.hits += 1
            return entry[0]
        self.misses += 1
        body = self.up.get(url, now) if isinstance(self.up, CacheTier) else self.up.get(url)
        self.items[url] = (body, now + self.ttl)
        self.items.move_to_end(url)
        if len(self.items) > self.cap:
            self.items.popitem(last=False)              # evict least recently used
        return body

    def purge(self, url):
        self.items.pop(url, None)


origin = Origin()
shield = CacheTier("shield", capacity=1000, upstream=origin, ttl=300)
pops = {}
for pop in ("fra", "lhr", "sin"):
    servers = [f"{pop}-{i}" for i in range(4)]
    pops[pop] = {s: CacheTier(s, capacity=50, upstream=shield, ttl=60) for s in servers}

urls = [f"/img/{i}.jpg" for i in range(30)]
now = 0
for rnd in range(5):                                    # users in 3 cities fetch popular images
    for pop, servers in pops.items():
        for i, url in enumerate(urls):
            for _ in range(1 + (30 - i) // 5):          # earlier URLs are more popular
                servers[pick_server(url, list(servers))].get(url, now)
    now += 20

edge_hits = sum(c.hits for s in pops.values() for c in s.values())
edge_miss = sum(c.misses for s in pops.values() for c in s.values())
print(f"edge hit ratio {edge_hits / (edge_hits + edge_miss):.1%} over {edge_hits + edge_miss} requests")
print(f"shield hits {shield.hits}, misses {shield.misses}; origin fetches {origin.fetches} (one per URL)")

# a server leaves the fra ring: only its share of URLs moves
before = {u: pick_server(u, list(pops["fra"])) for u in urls}
after = {u: pick_server(u, [s for s in pops["fra"] if s != "fra-2"]) for u in urls}
print("URLs remapped after removing fra-2:", sum(before[u] != after[u] for u in urls), "of", len(urls))

# purge: customer changes /img/0.jpg; invalidate everywhere instead of waiting for TTL
origin.version["/img/0.jpg"] = 2
shield.purge("/img/0.jpg")
for servers in pops.values():
    for c in servers.values():
        c.purge("/img/0.jpg")
print("after purge, sin serves:", pops["sin"][pick_server("/img/0.jpg", list(pops["sin"]))].get("/img/0.jpg", now))

Trade-offs

• Anycast gives instant failover and simple operations but less control over which PoP serves a user; DNS mapping steers precisely but reacts only as fast as DNS TTLs.
• Hashing objects to one owner per PoP multiplies effective cache capacity but adds an internal hop on most requests.
• A shield tier protects origins and raises the overall hit ratio but adds latency on misses and concentrates load on the shield PoPs.
• Long TTLs maximise hit ratio but rely on purges for freshness; short TTLs keep content fresh but send more traffic to origins.
• Lazy tag-based purges are cheap and fast to issue but leave stale bytes on disk until eviction and add a generation check to every hit.

What a strong answer shows

• Explains how users reach a nearby healthy PoP and compares DNS steering with anycast.
• Organises caches inside a PoP with hashing so capacity adds up, and handles very hot objects separately.
• Protects origins with a shield tier, request collapsing and stale-serving.
• Designs global purges with an ordered log, acknowledgements and lazy tag invalidation.
• Uses egress and hit-ratio numbers to justify decisions.
• Keeps the data plane running independently of the control plane.

Mistakes to avoid

• Treating the CDN as one big cache without explaining routing to the nearest location.
• Letting every edge server cache everything independently, wasting most of the PoP's capacity.
• Sending every edge miss straight to the origin, so a new release overloads it.
• Relying only on TTL expiry when customers need purges within seconds.
• Building the cache key from the whole URL including tracking parameters, destroying the hit ratio.

Follow-ups to expect

• How would you run customer code at the edge? (Lightweight isolates per request with strict CPU and memory limits, config pushed like zone settings.)
• How would you serve large video files efficiently? (Cache by byte range or segment, prefetch the next segments, keep popular titles in RAM.)
• How do you serve content that differs by device or language? (Add only the needed headers to the cache key via explicit rules, not by caching on every header.)
• How would you handle a DDoS against one customer? (Anycast spreads it across PoPs; rate limit and filter at the edge per zone.)

Practice next