When more than one agent helps
By the end: You can decide honestly whether to split a system and defend the decision.
- Why splitting can help
- Orchestrator and worker
- What it costs
- Measuring rather than assuming
New to multi-agent systems? Instead of one agent working through a whole task, you can split it up: one agent plans and hands bounded pieces of work to others, each with its own instructions, its own tools and its own separate conversation. A research assistant that sends three searchers off on different questions and combines what they bring back is a multi-agent system. Why it matters
Splitting one agent into several is the fashionable move and often the wrong one: it multiplies cost, latency and the number of places a run can go wrong. The honest reason it sometimes helps is context. A single agent that researches, then analyses, then writes carries all of it in one window, and by the end the material it needs is buried among everything else. This section covers when that trade pays, the orchestrator-and-worker shape it usually takes, what it costs you in tokens and in debugging, and how to measure the difference rather than assume it.
The default is one agent with more tools#
Start from the answer you will give most of the time, because it is the one most candidates get wrong. When someone proposes a second agent, what they almost always want is a capability the first agent does not have. A capability is a tool. Adding a tool costs you a function and a schema entry. Adding an agent costs you a second system prompt, a second transcript, a second budget, a handoff format, and a new question every time the output is wrong: which of them did this.
So the rule is: if the work needs a new ability, add a tool. Split into a separate agent only when you need something a tool cannot give you, and there are exactly three of those. You need a different context window, because the material genuinely does not fit or contaminates the rest. You need independent work to run at the same time, so three searches take the time of one. Or you need a different set of permissions, so the part reading untrusted text has no ability to act on it.
What you want What to add
another data source a tool
a different output format a prompt instruction
step-by-step reasoning nothing, ask for it
a specialist persona a prompt section, not an agent
work that must run in parallel an agent
a context that must stay clean an agent
untrusted text handled without tools an agent, with no toolsThe cost of getting this wrong is visible in a few lines. Below, the same job is done by one agent holding three tools and by a researcher handing notes to a writer. Watch what the writer posts.
# A minimal agent loop, so the comparison below runs rather than being a sketch.
class Agent:
def __init__(self, name, tools):
self.name = name
self.tools = {t.__name__: t for t in tools}
self.calls = 0
def run(self, plan):
self.calls += 1 # the call that picks the first tool
out = []
for tool_name, arg in plan:
if tool_name not in self.tools:
out.append("ERROR: " + self.name + " has no tool " + tool_name)
continue
out.append(self.tools[tool_name](arg))
self.calls += 1 # the call that reads the result
return " | ".join(out)
def search_docs(q): return "policy: refunds within 30 days"
def run_sql(q): return "sql: 412 refunds in March, 38 over policy"
def post_summary(s): return "posted: " + s[:44] # the channel truncates
RESEARCH = [("search_docs", "refund policy"), ("run_sql", "refunds in March")]
one = Agent("solo", [search_docs, run_sql, post_summary])
print(one.run(RESEARCH + [("post_summary", "38 refunds breached the 30-day policy in March")]))
# posted: 38 refunds breached the 30-day policy in Mar
print("model calls", one.calls) # model calls 4
researcher = Agent("researcher", [search_docs, run_sql])
writer = Agent("writer", [post_summary])
notes = researcher.run(RESEARCH)
print(writer.run([("post_summary", notes)]))
# posted: policy: refunds within 30 days | sql: 412 re
print("model calls", researcher.calls + writer.calls) # model calls 5The real reason it helps#
The benefit is not that several agents are cleverer than one. It is context. A single agent doing research, analysis and writing carries all of it in one window, and by the end the relevant material is buried among everything else.
Splitting gives each part a small focused context. The worker sees only its task; the orchestrator sees only summaries.
One agent: context holds 40k tokens of mixed research,
half irrelevant to the current step
Orchestrator + 3 workers:
each worker 5k tokens, only its own task
orchestrator 3k tokens, only the summariesThe common shape#
An orchestrator decomposes the task, dispatches bounded sub-tasks, and combines results. Workers do one thing and report back. Workers do not talk to each other, which keeps the system understandable.
import asyncio
class Budget:
def __init__(self, max_steps, max_usd):
self.max_steps, self.max_usd = max_steps, max_usd
self.steps, self.usd = 0, 0.0
def spend(self, in_tok, out_tok):
self.steps += 1
self.usd += in_tok / 1e6 * 1.0 + out_tok / 1e6 * 5.0
def exhausted(self):
return self.steps >= self.max_steps or self.usd >= self.max_usd
async def plan(task):
return ["pricing page", "changelog", "support tickets"]
async def run_worker(subtask, budget):
while not budget.exhausted():
budget.spend(1400, 250)
await asyncio.sleep(0.05) # stands in for the model call
if budget.steps >= 3: # this worker's own stopping rule
return "finding from " + subtask
return "incomplete: " + subtask # the budget ran out, so say so
async def synthesise(task, results):
return task + " -> " + " | ".join(results)
async def orchestrate(task):
subtasks = await plan(task)
results = await asyncio.gather(*(
run_worker(s, Budget(max_steps=5, max_usd=0.10)) for s in subtasks
))
return await synthesise(task, results)
print(asyncio.run(orchestrate("why did churn rise")))
# why did churn rise -> finding from pricing page | finding from changelog | finding from support tickets
async def starved():
return await run_worker("pricing page", Budget(max_steps=2, max_usd=0.10))
print(asyncio.run(starved()))
# incomplete: pricing pageWhat it costs#
- More calls, so more money. The orchestrator's planning and synthesis are extra.
- More latency unless workers run concurrently, and they often cannot if one depends on another.
- More failure modes: a worker fails, returns something useless, or misunderstands its brief.
- Harder debugging, because a wrong final answer might come from any worker or from the decomposition itself.
Decomposition is where it usually goes wrong
If the orchestrator splits the task badly, every worker does good work on the wrong question, and the synthesis is confidently wrong. This failure is harder to spot than a worker simply failing, because everything appears to have succeeded.
How much more it costs, actually#
People say multi-agent costs three times as much without ever working it out, and the honest answer is that it depends on one number: how much each worker writes back. Input tokens can go either way, because a single long conversation re-sends its whole history on every turn while a worker's context stays short. What never goes down is the number of calls, and what grows fastest is the output the supervisor has to read.
IN, OUT = 1.0, 5.0 # dollars per million tokens, illustrative
def single(turns=8, base=1200, growth=900, out=200):
calls = tin = tout = 0
ctx = base
for _ in range(turns):
calls += 1; tin += ctx; tout += out
ctx += growth # one tool result appended, re-sent next turn
return calls, tin, tout
def multi(workers=3, turns=3, sup=1000, w_base=800, growth=900, out=200, report=400):
calls = tin = tout = 0
calls += 1; tin += sup; tout += 300 # the plan
for _ in range(workers):
ctx = w_base
for _ in range(turns):
calls += 1; tin += ctx; tout += out
ctx += growth
tout += report # the worker writes its report
sup += report
calls += 1; tin += sup; tout += 100 # the supervisor reads it
calls += 1; tin += sup; tout += 600 # synthesis
return calls, tin, tout
def usd(t): return t[1] / 1e6 * IN + t[2] / 1e6 * OUT
for report in (200, 400, 800, 1600, 3200):
a, b = single(), multi(report=report)
print(report, a[0], round(usd(a), 4), b[0], round(usd(b), 4), round(usd(b) / usd(a), 2))
# report single calls single $ multi calls multi $ ratio
# 200 8 0.0428 14 0.0401 0.94
# 400 8 0.0428 14 0.0449 1.05
# 800 8 0.0428 14 0.0545 1.27
# 1600 8 0.0428 14 0.0737 1.72
# 3200 8 0.0428 14 0.1121 2.62Put your own numbers in before you defend either side in a review. The parameters that move the answer are the number of turns, how much context each turn adds, and the report length, and you can read all three off a week of traces.
Errors compound across handoffs#
Each handoff is a step that can go wrong, and the probabilities multiply. This is the arithmetic that makes long agent chains fail in a way that looks like bad luck but is not. A step that is right ninety-five times in a hundred sounds fine, and five of them in a row is a system that works three times in four.
for p in (0.99, 0.95, 0.90):
print(p, [round(p ** n, 3) for n in (1, 2, 3, 5, 8)])
# per-hop 1 hop 2 3 5 8
# 0.99 0.990 0.980 0.970 0.951 0.923
# 0.95 0.950 0.902 0.857 0.774 0.663
# 0.90 0.900 0.810 0.729 0.590 0.430The only defence is a check at the boundary rather than at the end. Validate what a worker returns against a schema before the supervisor believes it, and make an invalid return a retry of that one worker instead of a failure of the whole run. One retry takes a ninety-five per cent hop to 99.75 per cent, and three such hops end to end at 99.25 rather than 85.7, which is what makes chains of any length survivable.
Context is lost at every handoff, and it is silent#
A worker knows only what is in its brief. Everything the supervisor learned from the user, and everything it has already ruled out, exists only if somebody wrote it into that string. Nothing warns you when it did not, because the worker cheerfully answers the smaller question it was given.
state = { # what the supervisor knows
"question": "why did churn rise in March",
"segment": "enterprise accounts only",
"ruled_out": "plan tier, already checked, explains nothing",
}
brief = "Look into churn in March." # what it actually sent
def worker(brief):
if "enterprise" not in brief.lower():
return "Churn rose because self-serve trials expired."
return "Enterprise churn tracked the support backlog."
print(worker(brief))
# Churn rose because self-serve trials expired.
full = brief[:-1] + ", " + state["segment"] + ". Ruled out: " + state["ruled_out"]
print(worker(full))
# Enterprise churn tracked the support backlog.Measure it#
Run the same task set both ways. Compare success rate, cost and latency. Multi-agent is genuinely better on some workloads and worse on many, and the only way to know which you have is to try.
single multi
success 72% 81%
cost per task $0.14 $0.38
p95 latency 18s 31s
Better, at 2.7x the cost. Whether that is worth it is a
product decision, not a technical one.Interviewers ask about this directly, usually as whether your agents are separate services, tool-based or prompt-based. The strongest answer names the trade-off and says what you measured. The weakest one describes a diagram. If you have never run the comparison, say that, and say which of the three reasons above would make you run it.
What to skip#
Skip the frameworks until you have built the loop by hand once. Every orchestration library is the twenty lines above with a configuration format on top, and reaching for one first means you learn the format instead of the mechanism, then cannot debug it. Skip agents that exist to give a persona a name, because a section of the system prompt does that for nothing. Skip negotiation and voting protocols between agents entirely; they are interesting research and nobody is running them in production, and claiming one in an interview invites a question you cannot answer.
Where people go wrong
Adding agents because the architecture diagram looks more serious. Each one multiplies cost and failure modes, and reviewers will ask what you measured.
If this comes up in an interview
Why use multiple agents rather than one?
Mainly to keep each context small and focused. One agent doing everything ends up with the relevant material buried among everything else it has accumulated. It is a context management technique more than an intelligence one.
Then they push further
A colleague proposes five specialised agents for a support bot. What do you ask?
What one agent with good tools fails at, measured on evals. How context and decisions pass between agents and what is lost on the way. What it does to cost and latency. And how anyone will debug a wrong answer that passed through three agents.
What gets marked up. Marked up for treating several agents as a cost that must be justified by measured failures of one, with independent work that can run in parallel as the clearest case for it.
What does a multi-agent system cost you?
More calls and more money, more latency unless workers run concurrently, more failure modes, and harder debugging since a wrong answer can come from any worker or from the decomposition. Bad decomposition is the worst case, because every worker succeeds at the wrong task.
How would you decide between one agent and several?
Measure. Run the same task set both ways and compare success rate, cost and latency. Start with a single loop, because it is simpler and often sufficient, and split only when the numbers justify it.
Build this: Take a task your single agent handles poorly. Build an orchestrator and workers version, run both over twenty tasks, and report success, cost and latency for each.
Watch it explained
This is one section of Multi-agent systems, 10 hours across 2 sections. Read it in full to get the whole topic in order.
Stop 18 of 50 on the AI Engineer route
0 of 50 topics done · 0h behind you, 603h to go
What you have marked done is stored in this browser only. It does not reach a server, and clearing site data will clear it.