Asked about A structure for the design round

How do you handle the model being wrong in a production system?

An answer that works

Assume it will be. Show sources so users can verify, make correction cheap, keep a human in the loop where being wrong is expensive, gate destructive actions behind confirmation, and log everything so a bad answer can be traced. Design for wrong answers rather than trying to eliminate them.

Then they push further

Give a concrete example of that design.

For a refund assistant: the model drafts a decision with the policy it relied on, code checks the amount against the order and the policy limit, and anything over a threshold goes to a person. Users see the policy cited, so they can check it themselves.

What gets marked up

Marked up for designing on the assumption the model will sometimes be wrong: checks in code, human review where the cost is high, and errors that are visible and recoverable.

Reading an answer is not being able to give one. The section that teaches this is A structure for the design round, 5 hours of AI system design.

Learn this properly

Also asked about AI system design

Design a system that answers questions over a company's internal documents.How would you know the system had regressed?How do you handle document permissions in a RAG system?What would you build first?How do you stop a support agent issuing unauthorised refunds?How would you evaluate this agent?