Agent reliability interview questions
6 questions across 2 sections, in the order you would learn them. Every answer here is one that works — but an answer you have read is not an answer you can give, so use Test yourself and say it out loud first.
How agents fail
Read the section that teaches thisIf this comes up in an interview
What are the failure modes of an agent that ordinary software does not have?
Partial completion reported as success, confidently taking the wrong action, and compounding error where a small mistake early becomes an assumption later. None of these raise, so logs alone will not tell you.
How do you detect partial completion?
Do not trust the agent's summary. Check the world: whether each intended side effect actually happened. This is why agent evals assert on end state rather than on the text of the final answer.
Why does error compound in an agent?
Every turn is conditioned on all previous turns, so a wrong intermediate result becomes an assumption and then a conclusion. Without a human watching, nothing interrupts the chain, which is why shorter runs with checkpoints are safer than one long autonomous run.
Designing so failure is survivable
Read the section that teaches thisIf this comes up in an interview
How do you make an agent safe to retry?
Idempotent tools keyed on run id and intent, so a repeated call returns the previous result instead of acting twice. Checkpoint after each turn so a resumed run continues rather than restarting, and remember that the message list is not the whole state when a tool changed the world.
Then they push further
The payment API does not support idempotency keys. What do you do?
Record intent before calling: write a pending record with a unique id and check for it before any retry. Resolve uncertain cases by asking the payment system what actually happened instead of calling again.
What gets marked up. Strong answers mention idempotency keys and checking state before retrying, and treat a timeout as an unknown outcome rather than a failure.
Which agent actions need human approval?
Anything irreversible or externally visible: money, deletion, anything reaching a customer. Threshold-gate the rest by amount or volume. Gating everything defeats the purpose and gating nothing is how unreviewed actions happen.
How do you confirm an agent actually completed a task?
Verify against the world in code, not by asking the model. Check that each required side effect happened. Asking the agent uses the same process that produced any mistake, so it will report success either way.
These are the questions for one topic. The whole question bank covers every topic on the site.