Asked about Knowing whether it works
How do you evaluate an agent rather than a single completion?
An answer that works
Check the end state of the world rather than the text, because the agent's job is to do something. Then track efficiency alongside correctness: turns and cost per task. A run that succeeds after fourteen turns is not equivalent to one that succeeds in three.
Reading an answer is not being able to give one. The section that teaches this is Knowing whether it works, 4 hours of The agent loop.
Learn this properlyAlso asked about The agent loop
What is an agent, in one sentence?When would you not use an agent?Where does the loop run?How do you stop an agent running forever?An agent keeps calling the same tool with the same arguments. What do you do?What should an agent return when it runs out of budget?An agent overflows its context after ten turns. What do you do?What must survive compaction?
All 12 questions on The agent loop