Budgets, loops and giving up well
By the end: Your agent cannot run forever or spend unbounded money, and it explains why it stopped.
- Turn, time and cost limits
- Detecting a stuck agent
- Returning a partial result instead of raising
- Logging enough to reconstruct a run
New to the agent loop? Agent is a loaded word for something small. Call the model; if the reply asks to use a tool, run the tool, add the result to the conversation, and call the model again. Keep going until it stops asking or until you decide it has had enough turns. That loop, a few dozen lines of ordinary code, is the agent — there is no other machinery hiding behind the word. Why it matters
An agent decides for itself how many steps to take, so nothing in the loop you have just written prevents it running for hours. This section puts ceilings on a run, on turns, on time and on money, and covers what should happen when one is reached. Stopping well matters as much as stopping: a run that returns what it managed and says why it gave up is still useful to whoever asked, and an exception is not.
Three limits, not one#
A turn limit alone is not enough. A single turn can be slow, and a small number of turns with large contexts can be expensive. Track all three.
import time
class Budget:
def __init__(self, max_turns=10, max_seconds=120, max_usd=0.50):
self.max_turns = max_turns
self.deadline = time.time() + max_seconds
self.max_usd = max_usd
self.turns = 0
self.spent = 0.0
def exceeded(self) -> str | None:
if self.turns >= self.max_turns: return "turn limit"
if time.time() > self.deadline: return "time limit"
if self.spent >= self.max_usd: return "cost limit"
return NoneEveryone's first agent incident
An agent loops overnight, calling the same tool with the same arguments several hundred times, and the bill arrives before anyone notices. Add budgets before the first deployment, not after the first invoice.
What an unbounded run actually costs#
The reason a stuck run is expensive rather than merely slow is that the cost of a turn is not constant. Every turn resends the whole conversation, so the input grows on each one and the bill grows with the square of the turn count. Doubling the turns roughly quadruples the spend. The numbers below come from running the same stuck agent four times with different guards, priced at three dollars per million input tokens and fifteen per million output.
IN_PER_M, OUT_PER_M = 3.00, 15.00 # dollars per million tokens
def turn_cost(turn):
"""Every turn resends the whole conversation, so input grows each turn."""
tokens_in = 1200 + 700 * turn # system and tools, plus what has piled up
tokens_out = 120
return tokens_in / 1e6 * IN_PER_M + tokens_out / 1e6 * OUT_PER_M
class Budget:
def __init__(self, max_turns=10, max_usd=0.50):
self.max_turns, self.max_usd = max_turns, max_usd
self.turns, self.spent = 0, 0.0
def charge(self, turn):
self.spent += turn_cost(turn)
self.turns += 1
def exceeded(self):
if self.turns >= self.max_turns: return "turn limit"
if self.spent >= self.max_usd: return "cost limit"
return None
STUCK = [{"stop_reason": "tool_use",
"tool_calls": [{"name": "search", "args": {"q": "refund policy"}}]}]
class StubModel:
def __init__(self, script):
self.script, self.calls = list(script), 0
def chat(self, messages, tools=None):
self.calls += 1
return self.script[min(self.calls, len(self.script)) - 1]
def run_bounded(model, budget, detect_repeats=False, hard_cap=200):
last_sig, repeats = None, 0
for turn in range(hard_cap):
why = budget.exceeded()
if why:
return turn, why, budget.spent
reply = model.chat([])
budget.charge(turn)
if reply["stop_reason"] != "tool_use":
return turn + 1, "finished", budget.spent
call = reply["tool_calls"][0]
sig = (call["name"], tuple(sorted(call["args"].items())))
repeats = repeats + 1 if sig == last_sig else 0
last_sig = sig
if detect_repeats and repeats >= 2:
return turn + 1, "repeat detected", budget.spent
return hard_cap, "hard cap", budget.spent
for label, budget, detect in (
("no bounds at all", Budget(max_turns=200, max_usd=1e9), False),
("turn limit 10", Budget(max_turns=10, max_usd=1e9), False),
("cost limit $0.15", Budget(max_turns=200, max_usd=0.15), False),
("repeat detector", Budget(max_turns=200, max_usd=1e9), True),
):
turns, why, spent = run_bounded(StubModel(STUCK), budget, detect)
print(f"{label:<18}{turns:>4} turns ${spent:>6.3f} stopped on {why}")
# no bounds at all 200 turns $42.870 stopped on hard cap
# turn limit 10 10 turns $ 0.148 stopped on turn limit
# cost limit $0.15 11 turns $ 0.175 stopped on cost limit
# repeat detector 3 turns $ 0.022 stopped on repeat detectedTwo details in that table repay a second look. The cost limit stopped at 0.175 rather than 0.15, because a budget is checked after the turn that breached it and the last turn is always the most expensive one; if the overshoot matters, estimate the next turn from the current context size and refuse to start it. And the repeat detector cost less than a hundredth of the unbounded run, which is the general shape of this: budgets stop a disaster becoming a catastrophe, and noticing why the agent is stuck stops it being a disaster.
Spotting a stuck agent#
The classic failure is calling the same tool with the same arguments repeatedly, because the result did not help and the model has no other idea. Detect it and intervene.
import json
def repeat_guard(call, last_signature, repeats):
"""Returns the tool result to send back, or None to run the tool."""
signature = (call["name"], json.dumps(call["args"], sort_keys=True))
if signature == last_signature:
repeats += 1
if repeats >= 2:
return signature, repeats, {
"ok": False,
"error": "You have called this tool with identical arguments "
"three times. Try a different approach or stop."}
else:
repeats = 0
return signature, repeats, None
CALLS = [{"name": "search", "args": {"q": "refund policy"}}] * 4
last, repeats = None, 0
for i, call in enumerate(CALLS):
last, repeats, intervention = repeat_guard(call, last, repeats)
print(i, "repeats =", repeats, "->", intervention or "run the tool")
# 0 repeats = 0 -> run the tool
# 1 repeats = 1 -> run the tool
# 2 repeats = 2 -> {'ok': False, 'error': 'You have called this tool with identical arguments three times. Try a different approach or stop.'}
# 3 repeats = 3 -> {'ok': False, 'error': 'You have called this tool with identical arguments three times. Try a different approach or stop.'}Exact argument equality is the cheap version and it misses the common case, which is a loop that is stuck without repeating itself precisely: the same two tools alternating, or the same search with the query reworded each time. Two upgrades are worth the lines. Compare a canonical form of the arguments, lowercased with whitespace collapsed, so a reworded query still matches. And keep the last few signatures rather than one, so an A-B-A-B cycle is caught as well as A-A-A. The agent-reliability topic measures how much each of those catches on a mixture of stuck runs, because the answer is not obvious and it decides how much of this is worth building.
Whatever you detect, prefer telling the model over killing the run. A tool result saying it has now called this three times with the same arguments and should try something else costs one turn and often works, and if it does not, you still have the turn limit underneath.
The stop condition that never fires#
The other way a loop runs forever is subtler, and it is your bug rather than the model's. The loop stops when a condition is true, and the condition is written against the text instead of against the structure. A model told to end with the word DONE ends with the word DONE most of the time, which is the worst possible hit rate for a stop condition, because it works in testing.
FINALS = [
"DONE. The order has been refunded.",
"The order has been refunded.",
"I have refunded the order. Let me know if you need anything else.",
"Refund complete - task done.",
"All finished!",
]
def stops_on_text(text):
return "DONE" in text
def stops_on_stop_reason(reply):
return reply["stop_reason"] != "tool_use"
replies = [{"stop_reason": "end_turn", "text": t} for t in FINALS]
print("string check :", sum(stops_on_text(r["text"]) for r in replies), "/", len(replies))
print("stop_reason :", sum(stops_on_stop_reason(r) for r in replies), "/", len(replies))
# string check : 1 / 5
# stop_reason : 5 / 5Stop on the structured field the provider gives you, which the model cannot phrase differently. If the task genuinely needs a completion signal that the field cannot express, make it a tool the model calls, such as a finish tool taking the answer as an argument, so that finishing is a call you can validate rather than a phrase you have to match. The same argument applies anywhere you are tempted to parse intent out of prose the model wrote.
Stop usefully#
When a budget runs out, do not raise. Return what was achieved, what remains, and why it stopped. A partial answer with an explanation is useful to a person; an exception is not.
from dataclasses import dataclass
@dataclass
class AgentResult:
answer: str
complete: bool
stopped_because: str
turns_used: int
cost_usd: float
def give_up(best_so_far, why, turns_used, cost_usd):
return AgentResult(
answer=best_so_far,
complete=False,
stopped_because=why,
turns_used=turns_used,
cost_usd=cost_usd,
)
print(give_up("Found the order but could not confirm the refund.",
"cost limit", turns_used=11, cost_usd=0.175))
# AgentResult(answer='Found the order but could not confirm the refund.', complete=False, stopped_because='cost limit', turns_used=11, cost_usd=0.175)Log every turn#
Without a record of each turn you are guessing when something goes wrong. Log the tools requested, the arguments, the result size, the tokens and the cumulative cost. This is the beginning of tracing, and the first thing anyone asks about a misbehaving agent is what it actually did.
It is also, almost always, how you debug one. There is no debugger for a loop that did the wrong thing, because nothing went wrong in the ordinary sense: every tool returned successfully and the code did what it was told. What you have is a sequence of decisions, and the question is which one was the first bad one. So the log is not an operations nicety, it is the debugger, and it is worth designing as one.
import json
# One real run, logged. The customer asked about order 4471 and got a refund
# on a different order entirely.
LOG = [
{"turn": 0, "tool": "search_customer", "args": {"name": "bob"},
"result": {"matches": ["C-118", "C-402"]}, "tokens_in": 1200},
{"turn": 1, "tool": "list_orders", "args": {"customer": "C-402"},
"result": {"orders": ["A-9002"]}, "tokens_in": 1850},
{"turn": 2, "tool": "refund", "args": {"order_id": "A-9002", "amount_cents": 4500},
"result": {"ok": True}, "tokens_in": 2600},
]
def show(log):
print(f"{'turn':>4} {'tool':<16}{'arguments':<46}{'tokens_in':>9}")
for e in log:
print(f"{e['turn']:>4} {e['tool']:<16}{json.dumps(e['args']):<46}{e['tokens_in']:>9}")
def first_appearance(log, value):
"""Where did the agent get this from? The single most useful query."""
for e in log:
if value in json.dumps(e["result"]):
return e["turn"], e["tool"], e["args"]
return None
show(LOG)
print()
print("the refund used order", LOG[-1]["args"]["order_id"])
print("first seen at turn", first_appearance(LOG, "A-9002"))
print("that came from customer C-402, and turn 0 returned two matches:",
LOG[0]["result"]["matches"])
# turn tool arguments tokens_in
# 0 search_customer {"name": "bob"} 1200
# 1 list_orders {"customer": "C-402"} 1850
# 2 refund {"order_id": "A-9002", "amount_cents": 4500} 2600
#
# the refund used order A-9002
# first seen at turn (1, 'list_orders', {'customer': 'C-402'})
# that came from customer C-402, and turn 0 returned two matches: ['C-118', 'C-402']The bug is now obvious and it is not in turn two. Turn zero returned two customers called Bob and the model picked one, and nothing in the loop noticed that a lookup meant to identify one person returned two. The fix is in the tool, not the prompt: a search that matches more than one record should return an error saying so rather than a list, because a list invites a guess. That is the usual shape of an agent bug. The turn where the damage happened is rarely the turn where the mistake was made.
- Log the arguments in full, not a summary. The wrong argument is the bug most of the time.
- Log the result, truncated with the original length recorded, so you can tell a tool returning nothing from a tool returning ten thousand tokens.
- Log tokens in, tokens out and cumulative cost per turn, or you cannot answer why a run cost what it did.
- Give every run an id and put it in every log line, because you will be reading two interleaved runs sooner than you think.
- Keep the exact system prompt and tool definitions with the run. A run you cannot reproduce because the prompt has moved on is an anecdote.
One habit is worth more than all of them: before changing anything, read the transcript to the end and say out loud what you think the agent believed at each turn. Most agent bugs that survive an afternoon are cases where the engineer fixed the turn that looked wrong instead of the turn that made it inevitable.
Where people go wrong
Only limiting turns. A three-turn run with a very large context can cost more than a twenty-turn run with small ones, so a turn limit gives you no real protection against spend.
If this comes up in an interview
How do you stop an agent running forever?
Three independent budgets: turns, wall-clock time and cost. Any one being exceeded stops the run. Turns alone are not enough because a single turn can be slow or expensive.
Then they push further
Why is a turn limit alone not enough?
One turn can call a slow tool or produce a huge output, so a run can blow its time or cost budget in very few turns. Each budget protects against a different failure.
What does the user see when a budget trips?
A structured partial result: what was done, what was not, and why it stopped, with a way to continue if that makes sense. A silent failure or a generic error throws away the work already done.
How would you choose the limits?
From data. Run the eval tasks, look at turns, time and cost for the runs that succeeded, and set limits a little above the high end of that range. Revisit them when tools or models change.
What gets marked up. Marked up for independent budgets, for returning partial results instead of throwing, and for deriving limits from measured runs rather than picking round numbers.
An agent keeps calling the same tool with the same arguments. What do you do?
Detect the repeat by hashing the tool name and arguments, and when it repeats, return a result telling the model it is stuck and should try something else. Killing the run is the fallback if that does not work.
What should an agent return when it runs out of budget?
A structured partial result: what it achieved, what it could not, why it stopped, and what it cost. Raising an exception throws away work the caller could still use.
Build this: Add all three budgets to your loop, then give the agent an impossible task and confirm it stops on each limit in turn by tightening one at a time.
Watch it explained
This is one section of The agent loop, 14 hours across 4 sections. Read it in full to get the whole topic in order.
Stop 16 of 50 on the AI Engineer route
0 of 50 topics done · 0h behind you, 603h to go
What you have marked done is stored in this browser only. It does not reach a server, and clearing site data will clear it.