Asked about Designing for a component that is sometimes wrong
Would you show a confidence score to users?
An answer that works
Only if it is calibrated, meaning that things marked 80 percent are right about 80 percent of the time. An uncalibrated number is worse than none, because it looks like information and misleads.
Reading an answer is not being able to give one. The section that teaches this is Designing for a component that is sometimes wrong, 5 hours of Product judgement for AI.
Learn this properlyAlso asked about Product judgement for AI
How do you design a feature around a model that is sometimes wrong?Why are AI failures harder for users than ordinary software failures?How do you decide whether a model is accurate enough to ship?You launched an AI writing assistant. How do you know it is helping?How do you decide what to fully automate?