Design a Recommendation Feed

Design the personalised home feed for a large consumer app with a catalog of about 50 million items (short videos, posts and products from millions of creators). 100 million people open the app every day and scroll through a ranked list chosen just for them.

Scoring every item for every user on every request is impossible, yet the feed must react within minutes when someone starts liking a new topic, and brand-new items must get a chance to be seen. The business wants more meaningful engagement, not just clicks, and every change to the models must be proven in an experiment before it ships.

In 45 minutes, cover the serving funnel (candidate generation, ranking, re-ranking), embeddings and the two-tower model, features and the feature store, the online and offline data pipelines, cold start, and how you run A/B tests on the feed.

Sketch on the HLD board, write your answer in the Design doc, and size it with the Estimator. Start the mock interview from the timer for a 45-minute run.

Design Path · AI & ML Systems · problem 51 of 70

Ask before you design. Try answering each yourself, then compare.

1. How big is the catalog and how fast does it change?
   Typical answer

   ~50 M active items; ~2 M new items per day. New items should be recommendable within minutes.

2. What traffic?
   Typical answer

   100 M DAU, ~20 feed page loads each, so ~2 B requests/day; ~70k/s at peak.

3. What latency?
   Typical answer

   p99 under 200 ms for a page of 20 items, server side.

4. What is the objective?
   Typical answer

   A weighted mix: probability of a long view, like, share and follow, minus hides and reports. Weights are a product decision.

5. What signals do we have?
   Typical answer

   Impressions, clicks, watch time, likes, shares, hides, follows, plus item content and creator metadata. ~5 B events/day.

6. Any business rules?
   Typical answer

   No more than two items from one creator per page, nothing the user has already seen recently, and integrity filters for removed content.

Requirements

Functional

• Return a personalised, paginated feed for a user.
• Reflect new user interactions within minutes.
• Recommend new items within minutes of upload (cold start).
• Apply business rules: diversity, seen-filtering, integrity removals.
• Run concurrent A/B experiments on retrieval, ranking and rules.

Non-functional

• p99 < 200 ms server-side; ~70k requests/s peak.
• 99.95% availability; degrade to cached or popular feeds rather than error.
• Feature freshness: streaming features < 1 min; models retrained at least daily.
• Ingest ~5 B events/day (~175k/s peak) without loss for training.

Out of scope

• Search and ads ranking.
• Content moderation models themselves.
• Creator analytics.

Back-of-the-envelope estimates

Here a "write" is an interaction event and a "read" is a feed request.

• Feed requests: 100 M × 20 = 2 B/day ≈ 23k/s average, ~70k/s peak.
• Brute force is impossible: 70k × 50 M items = 3.5 trillion scorings/s. Retrieval must cut 50 M to ~1,000 per request.
• Ranking load: 70k × 1,000 = 70 M candidate scorings/s. Assuming a GPU scores ~500k candidates/s with a mid-size model, that is ~140 GPUs; a cheap first-pass ranker that trims to ~300 cuts it a lot.
• Item index: 50 M × 128 dims × 4 B ≈ 25 GB (~6 GB with int8): fits in memory on every retrieval replica.
• Events: 5 B/day ≈ 58k/s average, ~175k/s peak; at ~200 B each that is ~1 TB/day of raw logs.
• Features: ~100 M users × ~2 KB + 50 M items × ~1 KB ≈ 250 GB in the online store.

The 50 M-item catalog and the 200 ms budget drive the funnel design; the event volume drives the streaming and training pipelines.

API

GET  /v1/feed?cursor=...&limit=20          -> {items:[{itemId, reason?, trackingToken}], nextCursor}
POST /v1/events   [{type: impression|click|view|like|hide, itemId, trackingToken,
                    dwellMs?, ts}]                    (batched by the client)

// internal
Retrieve(userId, context, k)          -> candidates with source tags
Rank(userId, candidates, experimentArms) -> scores per objective

Every served item carries a tracking token that records request id, model versions and experiment arms, so later events join back to exactly what was shown and why, which is the backbone of both training labels and experiment analysis.

Data model

Online feature store (wide-column key-value, keyed by user_id or item_id): user features (recent engagement by topic, long-term embedding, counts per action type over 1 h / 1 d / 28 d) and item features (CTR by slice, age, creator stats). It must answer ~1,000 item lookups per request in a few ms, so values are compact and partitioned by id.

ANN item index: item_id → 128-d vector plus filter attributes (language, region, removed flag), rebuilt from each new item tower and updated incrementally as items are created. Seen cache (Redis): recent impressions per user. Data lake: event logs joined with the feature values logged at serving time, partitioned by date, which become training examples. Model registry (object storage plus metadata): versioned models and embeddings with offline metrics.

High-level design

Read path: the app asks the feed API for a page. The API gets the user's experiment arms from the experiment service (a pure hash, cached) and calls the feed service. Candidate generation runs several sources in parallel: the user tower turns fresh user features into a vector and the ANN item index returns the nearest ~1,000 items, while smaller sources add items from followed creators, trending items and fresh uploads reserved for exploration. Candidates already in the seen cache or removed for integrity are dropped. The ranking service fetches user and item features from the online feature store and runs a multi-task model that predicts each objective; the feed service combines them with product weights, then re-ranks for diversity and business rules and returns 20 items with tracking tokens.

Write path: the app batches impressions and actions to the event log. Streaming features jobs update counters and recent interests in the feature store within seconds, so the very next page reflects a new interest. Events are also archived to the data lake, joined with features logged at serving time, and the training pipeline retrains the two-tower and ranking models daily (with hourly incremental updates). New models and item embeddings go to the model registry, are validated offline, and roll out to the ranker and ANN index behind an experiment.

Open the HLD board and press Reference to see the architecture diagram.

Deep dives

Two-tower candidate generation

A user tower and an item tower each map their features to a 128-d vector, trained so the dot product is high for items the user engaged with. Training uses positives from engagement logs and negatives from other items in the batch, with a correction for the fact that popular items appear as negatives more often. Because the towers are separate, item vectors are precomputed and indexed for approximate nearest-neighbour search, while the user vector is computed per request from fresh features in about a millisecond. One embedding source is not enough: blend in followed-creator, trending, item-to-item ("similar to what you just liked") and fresh-content sources with per-source quotas, so the ranker sees variety and new items.

Ranking and training-serving skew

The ranker sees far richer features than retrieval: user–item cross features, creator affinity, recency, context like time of day and device. A multi-task network predicts several outcomes (long view, like, share, hide) from shared layers; the final score is a weighted sum chosen by product, so weights can change without retraining. The most common silent bug is skew between how a feature is computed for training and how it is computed for serving. Avoid it by logging the exact feature values used at serving time with the tracking token and training on those logs, and by defining features once for both the streaming and batch pipelines. A light first-stage ranker trims 1,000 candidates to a few hundred to save GPU time.

Freshness, cold start and exploration

Three clocks: streaming features update in seconds (what you just watched), item embeddings for new uploads come from a content-only tower within minutes so a new item can be retrieved before it has any clicks, and full models retrain daily with hourly incremental updates. Cold-start items get a small reserved exploration slot on a page, allocated by a bandit that boosts items with promising early engagement and stops showing poor ones quickly. New users start from onboarding choices, location and popular-in-region items, and the user tower adapts after a handful of interactions because it reads recent engagement features directly.

A/B testing the feed

Assign users to arms by hashing experiment_id:user_id, which is deterministic and needs no lookup; use layers so retrieval, ranking and UI experiments can run at once on independent hashes. Pick a primary metric that reflects long-term value (days active, meaningful engagement time) plus guardrails (hides, reports, latency, creator diversity). Watch for novelty effects in the first days, run long enough to cover weekly cycles, and keep a small long-term holdout that receives no new launches to measure cumulative impact. Offline metrics such as AUC or recall@k are necessary filters before an experiment, but they often fail to predict online results, so they gate experiments rather than replace them.

Failure modes

• If the ranking service is slow or down, the feed service serves retrieval order with simple heuristics, or a precomputed feed cached per user.
• If the feature store is degraded, missing features fall back to defaults the model was trained to handle, and timeouts are short.
• If the streaming pipeline lags, recommendations go stale but stay reasonable; alert on event-to-feature lag.
• If a bad model ships, experiment guardrails catch it on a small slice; the registry rolls back to the previous version.
• If the event log loses data, training labels are biased; use acknowledged, replicated logging and monitor event counts against served impressions.

Core algorithm in code

import hashlib
import math
import random

rng = random.Random(3)
DIM = 8
TOPICS = ["cooking", "football", "travel", "music"]
topic_vec = {t: [rng.gauss(0, 1) for _ in range(DIM)] for t in TOPICS}

# Item tower output (precomputed offline and loaded into an ANN index).
items = {}
for i in range(400):
    t = TOPICS[i % 4]
    items[i] = {"topic": t, "creator": i % 23, "age_h": rng.randint(1, 240),
                "ctr": rng.uniform(0.01, 0.12),
                "vec": [x + rng.gauss(0, 0.6) for x in topic_vec[t]]}


def user_tower(history):
    """User vector = mean of recently engaged item vectors (stand-in for a learned tower)."""
    vs = [items[i]["vec"] for i in history]
    return [sum(c) / len(vs) for c in zip(*vs)]


def dot(a, b):
    return sum(x * y for x, y in zip(a, b))


def retrieve(user_vec, item_vecs, k):
    return sorted(item_vecs, key=lambda i: -dot(user_vec, item_vecs[i]))[:k]


def rank(user_vec, candidates):
    out = []
    for i in candidates:
        it = items[i]
        z = (0.35 * dot(user_vec, it["vec"]) + 25 * it["ctr"]
             - 0.01 * it["age_h"] - 2.0)              # hand-set weights stand in for a trained model
        out.append((1 / (1 + math.exp(-z)), i))
    return sorted(out, reverse=True)


def rerank(scored, max_per_creator, n, seen):
    page, per_creator = [], {}
    for p, i in scored:
        c = items[i]["creator"]
        if i in seen or per_creator.get(c, 0) >= max_per_creator:
            continue
        per_creator[c] = per_creator.get(c, 0) + 1
        page.append((round(p, 3), i, items[i]["topic"], c))
        if len(page) == n:
            break
    return page


def bucket(user_id, experiment, buckets=100):
    h = hashlib.sha256(("%s:%s" % (experiment, user_id)).encode()).hexdigest()
    return int(h, 16) % buckets


history = [0, 4, 8, 12, 2]                 # mostly cooking, one travel item
uvec = user_tower(history)
cands = retrieve(uvec, {i: it["vec"] for i, it in items.items()}, k=60)
print("retrieved %d of %d items; topics: %s" % (len(cands), len(items),
      {t: sum(items[i]["topic"] == t for i in cands) for t in TOPICS}))
page = rerank(rank(uvec, cands), max_per_creator=1, n=6, seen=set(history))
for row in page:
    print("  p(engage)=%.3f item=%d topic=%s creator=%d" % row)

arms = [("control" if bucket(u, "ranker_v7") < 50 else "treatment") for u in range(10000)]
print("A/B split:", arms.count("control"), "control /", arms.count("treatment"), "treatment")

Trade-offs

• A two-tower retriever is fast and scales to the whole catalog but cannot model fine user-item interactions, which the ranker must add.
• More candidates per request improve recall but cost ranking GPU time and latency.
• Streaming features make the feed responsive within seconds but cost a complex real-time pipeline and consistency work with batch features.
• Exploration slots give new items a chance but cost some short-term engagement.
• Optimising a blended objective aligns with long-term value but is harder to tune and explain than a single click-through target.

What a strong answer shows

• Uses numbers to justify a multi-stage funnel.
• Explains two-tower retrieval and why item vectors can be precomputed.
• Handles training-serving skew with logged features and shared definitions.
• Separates online and offline pipelines with clear freshness for each.
• Has a cold-start and exploration strategy.
• Designs experiments with sensible metrics, guardrails and pitfalls.

Mistakes to avoid

• Proposing to score the whole catalog with one big model per request.
• Collaborative filtering only, with no answer for new items or new users.
• Optimising only for clicks and ending up with clickbait.
• Training on features recomputed later instead of the values seen at serving time.
• Judging models only by offline metrics.

Follow-ups to expect

• How would you avoid feedback loops that only show popular content? (Exploration, debiasing for position and popularity.)
• How do you add ads into the feed? (A separate auction-ranked list merged with spacing rules.)
• How would you explain a recommendation to the user? (Record the candidate source and top features with the tracking token.)
• How would you serve users in a new country with little data? (Regional popularity, multilingual content embeddings, faster exploration.)

Practice next