Asked about Tracking that people actually use
A model in production is behaving differently from what the team remembers testing. How do you find out what was trained?
An answer that works
Look up the model's run record: code commit, data version, config and metrics. If those were not logged, that is the finding and the fix. Then compare that run's evaluation with current production behaviour to see whether the model or the data changed.
Then they push further
The run record shows a data version, but that table has since been overwritten. Now what?
The run cannot be reproduced exactly. For the future, snapshot or version training data, with immutable files or a data versioning tool, and log the snapshot id.
What gets marked up
Marked up for naming code, data, config and environment, and for treating data versioning as the usual gap.
Reading an answer is not being able to give one. The section that teaches this is Tracking that people actually use, 3 hours of Experiment tracking.
Learn this properlyAlso asked about Experiment tracking