Asked about Measuring and reducing what could go wrong
Why is a negative safety result weak evidence?
An answer that works
Because failure to elicit is not absence of capability. Better prompting, tool access or more attempts may succeed where your evaluation did not, so the claim is always bounded by how hard you tried.
Reading an answer is not being able to give one. The section that teaches this is Measuring and reducing what could go wrong, 5 hours of Safety & evaluation research.
Learn this properlyAlso asked about Safety & evaluation research
How do you evaluate whether a model has a dangerous capability?What is red teaming?How do you make a safety evaluation's negative result meaningful?Why evaluate the system rather than the model?How would you handle disclosing a vulnerability you found?Your evaluation finds a model cannot complete a concerning task. What can you conclude?How would you red team an LLM application before launch?How would you test whether a model's chain of thought is faithful?