Asked about Keeping a model working after launch
What is model drift and how do you detect it?
An answer that works
Data drift is the inputs changing; concept drift is the relationship between inputs and outputs changing. Detect it by monitoring input and prediction distributions against a fixed reference, since labels often arrive late or never, so accuracy is not available in real time.
Then they push further
Your AUC has not moved in six months. Does that mean nothing has drifted?
No. AUC depends only on the ranking, so it is invariant to any monotone rescaling of the scores and cannot see calibration drift at all. A base rate that moves from 10 per cent to 40 per cent leaves AUC identical to four decimal places while the predicted probabilities become wrong by a quarter, which breaks every expected-value decision downstream. Watch mean predicted probability against realised rate as well as a ranking metric.
Why compare against a fixed reference rather than against last week?
Because a distribution moving one per cent a day never trips a week-on-week comparison and has moved thirty per cent by the end of the month. Hold the reference fixed, usually the validation set the model was signed off on, and move it deliberately when you retrain.
What gets marked up
Marked up for naming both kinds, for monitoring outputs rather than waiting on labels, and for knowing which failures the usual metrics cannot see.
Reading an answer is not being able to give one. The section that teaches this is Keeping a model working after launch, 5 hours of MLOps & model lifecycle.
Learn this properlyAlso asked about MLOps & model lifecycle