Asked about Knowing whether it works
Your agent's success rate is unchanged after a prompt edit. Is it safe to ship?
An answer that works
Not necessarily. Check turns and cost as well. A change can hold the pass rate while doubling spend, and nothing in a pass-rate-only eval will tell you.
Reading an answer is not being able to give one. The section that teaches this is Knowing whether it works, 4 hours of The agent loop.
Learn this properlyAlso asked about The agent loop
What is an agent, in one sentence?When would you not use an agent?Where does the loop run?How do you stop an agent running forever?An agent keeps calling the same tool with the same arguments. What do you do?What should an agent return when it runs out of budget?An agent overflows its context after ten turns. What do you do?What must survive compaction?
All 12 questions on The agent loop