Asked about Designing for a component that is sometimes wrong
How do you decide whether a model is accurate enough to ship?
An answer that works
By comparing time saved against time spent, not by picking a threshold. The cost has three parts: checking every answer, redoing the ones the check catches, and the incidents caused by the ones it misses. On a twelve-minute task with a two-and-a-half-minute check, a check that catches 85 per cent of errors needs about 80 per cent model accuracy to break even; improve the check to 99 per cent and the bar falls to about 38 per cent. So the first lever is the check, not the model.
Then they push further
When is the answer that you should not build it?
When the cost of an error that escapes is high enough that break-even accuracy goes above anything achievable, or when the user cannot tell the output is wrong, or when the action is irreversible. Then it is a draft behind an approval step, or nothing.
What gets marked up
Strong answers produce a number and treat verification as the variable they control. Promising to evaluate carefully is not an answer.
Reading an answer is not being able to give one. The section that teaches this is Designing for a component that is sometimes wrong, 5 hours of Product judgement for AI.
Learn this properlyAlso asked about Product judgement for AI