Asked about Testing a system with a non-deterministic part

How do you test code that calls an LLM?

An answer that works

Fake the model at the boundary with scripted replies, which makes everything around it deterministic: parsing, retries, budgets, tool dispatch, prompt assembly. The quality of the model's output is not a unit test; that is what evals are for.

Reading an answer is not being able to give one. The section that teaches this is Testing a system with a non-deterministic part, 6 hours of Testing.

Learn this properly

Also asked about Testing

What is usually untested in an AI system and should not be?What is the difference between a test and an eval?How do you test an agent without calling a model?What is property-based testing and where does it help in AI code?