Asked about Build a recommender end to end
How would you evaluate a recommender offline?
An answer that works
A time-based split so the model never sees the future, ranking metrics such as recall at k and NDCG against each user's later interactions, and comparison against a tuned popularity baseline. Then treat offline results as a filter and confirm with an online test, since offline data reflects what the old system chose to show.
Then they push further
Why is offline evaluation biased?
Users could only interact with what the previous system showed them. Items it never surfaced have no interactions, so a new model that recommends them looks wrong offline even if users would love them.
What gets marked up
Marked up for time-based splits, a popularity baseline, and naming exposure bias.
Reading an answer is not being able to give one. The section that teaches this is Build a recommender end to end, 6 hours of Recommendation systems.
Learn this properlyAlso asked about Recommendation systems