The recommender system design interview
By the end: You can lead a system design discussion for a feed or recommendation product from requirements to experiments.
- Clarifying the objective
- Candidate generation from several sources
- Ranking and re-ranking
- Serving within a latency budget
- Logging, feedback and experiments
New to recommendation systems? A recommender chooses the handful of items to put in front of one person, out of a catalogue of millions, in a few milliseconds. It does it in two stages: retrieve a few hundred plausible candidates cheaply, then rank those carefully. The signals are what this person did before and what people who behaved similarly did next. Why it matters
A feed or a recommendation product is a standard system design question, and it has a standard shape: cut millions of candidate items down to a few hundred cheaply, rank those carefully, re-rank for the things the ranker cannot see, and do all of it inside a latency budget. This section walks that structure from the objective down to logging and experiments. The objective comes first, because watch time and long-term retention lead to genuinely different systems.
Start with the objective#
Ask what success means before drawing boxes. Watch time, purchases, long-term retention and user satisfaction lead to different systems. Ask about scale: users, items, requests per second, and how fast new items appear.
If the objective is the system leans towards
watch time long items, autoplay, strong session modelling
purchases price and stock features, heavy re-ranking on margin
retention over months diversity and exploration, engagement weighted down
subscriptions breadth of catalogue surfaced, not depth of one taste
creator ecosystem health an impression floor for new creators, by designThe shape of the answer#
request
-> candidate generation (several sources, ~1,000 items, tens of ms)
two-tower ANN retrieval | recent interactions | trending | followed creators
-> ranking (rich features, a heavier model, ~1,000 -> ~100)
-> re-ranking (diversity, freshness, business rules, remove seen items)
-> response
-> logging impressions, clicks, dwell, skips -> training data for tomorrowPut a number on each arrow. A feed request that must return in 200 milliseconds at the ninety-ninth percentile does not have 200 milliseconds of model time in it, and being able to say where the budget goes is what separates a design answer from a diagram.
fetch user features from the cache 5 ms
user tower forward pass 3 ms
ANN search over the item index 8 ms -- and the other sources, parallel
fetch features for ~1,000 candidates 25 ms -- one batched call, never 1,000
ranking model over 1,000 candidates 40 ms
re-ranking and business rules 5 ms
------
86 ms leaving room for the network,
serialisation and one retryRe-ranking, and why it is a separate stage#
The ranker scores each item on its own, so it cannot see that the top ten are nearly the same item. Re-ranking is where the list is judged as a list: diversity, freshness, not showing the same creator four times, honouring whatever rules the business has. Maximal marginal relevance is the standard mechanism, and it is short enough to write out.
import numpy as np
def mmr(scores, similarity, k=5, lam=0.7):
"""Pick items that are good and unlike what has already been picked.
lam=1 is pure relevance; lam=0 is pure diversity."""
chosen = []
remaining = list(range(len(scores)))
while remaining and len(chosen) < k:
best, best_value = None, -np.inf
for i in remaining:
penalty = max((similarity[i][j] for j in chosen), default=0.0)
value = lam * scores[i] - (1 - lam) * penalty
if value > best_value:
best, best_value = i, value
chosen.append(best)
remaining.remove(best)
return chosen
scores = np.array([0.90, 0.89, 0.88, 0.60, 0.55])
# The first three are near-duplicates of each other; the last two are unrelated.
similarity = np.array([
[1.0, 0.95, 0.93, 0.05, 0.02],
[0.95, 1.0, 0.94, 0.03, 0.04],
[0.93, 0.94, 1.0, 0.02, 0.03],
[0.05, 0.03, 0.02, 1.0, 0.10],
[0.02, 0.04, 0.03, 0.10, 1.0],
])
print(mmr(scores, similarity, k=3, lam=1.0)) # [0, 1, 2] three of the same thing
print(mmr(scores, similarity, k=3, lam=0.7)) # [0, 3, 4] one of eachThe honest tradeoff is that diversity costs engagement today and buys it back over months, and the second half of that sentence is far harder to measure than the first. A two-week test will usually show diversity losing. That is the argument for holdback groups that run for a quarter, and it is also why this decision is made by whoever owns the product rather than by whoever owns the metric.
Logging, or there is no tomorrow#
The training data for next month's model is what today's system writes down, so the logging schema is a modelling decision, not an operational detail. Log the impression, not only the click: an item shown and ignored is the negative example the ranker needs, and a system that logs only clicks can never learn what people skip.
per impression request_id, user_id, item_id, position, timestamp
the model version and the ranking score it gave
the candidate source that produced it
the feature values as used, not recomputed later
per outcome click, dwell seconds, save, share, hide, report
joined back on request_id and item_idPoints interviewers look for#
- Several candidate sources, so one weak source does not starve the feed.
- The ranking objective as a combination of predicted actions, such as click, long watch and share, with weights tied to the business goal.
- Logging impressions, not only clicks, so non-clicks become negatives.
- Cold start for new users and new items, with exploration.
- Feedback loops: the model trains on what it chose to show. Some exploration keeps the data honest.
- Online experiments with guardrails, and a time-based offline evaluation.
Where people go wrong
Starting with the ranking model architecture before asking what the product is trying to optimise.
If this comes up in an interview
Design the recommendation system for a short-video app's home feed.
Clarify the objective and scale. Then describe multi-source candidate generation, including two-tower retrieval with approximate nearest neighbour search, a ranking model predicting several engagement signals combined by weights tied to the goal, re-ranking for diversity and freshness, logging of impressions and outcomes, cold-start handling, exploration, and evaluation with time-split offline metrics and online A/B tests.
Then they push further
Engagement went up but users report the feed feels repetitive. What do you change?
Add diversity in re-ranking, penalise near-duplicates and over-represented creators, include longer-term signals such as return visits in the objective, and track a diversity metric and user surveys as guardrails.
How do you handle a brand-new video with no interactions?
Embed it from content features such as visuals, audio, text and creator, give it a small exploration budget of impressions, and let early engagement quickly update its ranking.
What gets marked up. Strong answers start from the objective, separate retrieval from ranking, and address feedback loops and cold start without prompting.
Build this: Write a two-page design doc for recommendations in a product you use: objective, scale, candidate sources, ranking signals, re-ranking rules, logging and the first experiment you would run.
Watch it explained
This is one section of Recommendation systems, 19 hours across 4 sections. Read it in full to get the whole topic in order.