Coordination patterns and their costs

Once one agent works, the obvious move is more of them, and there are only a handful of shapes they can be arranged in. This section names them, an orchestrator with workers, a fixed pipeline, a producer with a critic, several attempts in parallel, and gives the cost of each, since every one multiplies the calls and adds a way to fail. It also covers shared state, which is where multi-agent systems reproduce every concurrency bug you have already met elsewhere.

Common shapes#

• Orchestrator and workers: one agent plans and dispatches, workers execute bounded tasks and report back. The most common and the easiest to reason about.
• Pipeline: fixed stages, each handled by a specialised agent. Predictable, and only suits tasks whose steps are known in advance.
• Critic loop: one agent produces, another reviews and requests changes. Improves quality at the cost of extra calls and the risk of endless revision.
• Parallel exploration: several agents attempt the same problem differently and the best result is chosen.

The three that matter in practice are below, in code, each short enough to hold in your head. They differ in one thing: who decides what happens next. The supervisor decides at runtime, the pipeline decided at design time, and the parallel fan-out does not decide at all because nothing depends on anything else. That single question tells you which one you have, and it is the question to answer out loud before drawing anything.

Are the steps known before you read the task?
    yes, and each feeds the next        -> pipeline
    yes, and they are independent       -> parallel fan-out
    no, they depend on the task         -> supervisor and workers

Does the output need judging as well as producing?
    yes, and quality matters more than cost  -> add a critic, capped
    otherwise                                -> do not

Supervisor and workers#

One agent turns the task into a list of bounded sub-tasks, runs each one, and merges what comes back. Workers never talk to each other, which is the property that keeps it debuggable: every result has exactly one producer and exactly one consumer. Reach for this when the sub-tasks are not known until you have read the task.

# FakeClient stands in for the model so the control flow is runnable.
# In real code, chat() is client.messages.create.
class FakeClient:
    def __init__(self):
        self.calls = 0
    def chat(self, messages):
        self.calls += 1
        prompt = messages[-1]["content"]
        if prompt.startswith("PLAN"):
            return "pricing page\nchangelog\nsupport tickets"
        if prompt.startswith("WORK"):
            return "finding from " + prompt.split(": ", 1)[1]
        return "combined: " + prompt.split(": ", 1)[1].replace("\n", " | ")

client = FakeClient()

def supervise(task):
    plan = client.chat([{"role": "user", "content": "PLAN the task: " + task}])
    subtasks = [line for line in plan.splitlines() if line.strip()]
    findings = [client.chat([{"role": "user", "content": "WORK on: " + s}])
                for s in subtasks]
    return client.chat([{"role": "user", "content": "MERGE these: " + "\n".join(findings)}])

print(supervise("why did churn rise in March"))
# combined: finding from pricing page | finding from changelog | finding from support tickets
print("model calls:", client.calls)      # model calls: 5

Pipeline#

Fixed stages in a fixed order, each stage taking the previous stage's output. There is no planning call and no merge, so it is the cheapest shape there is, and it only applies when you already know every step. The real reason to use it is not cost but that you can put an ordinary function between two model calls, and validation is an ordinary function.

def extract(doc):     return {"amount": 420, "currency": "EUR", "vendor": "Acme"}
def validate(rec):
    if rec["amount"] <= 0:
        raise ValueError("amount must be positive")
    return rec
def summarise(rec):   return rec["vendor"] + " invoiced " + str(rec["amount"]) + " " + rec["currency"]

STAGES = [extract, validate, summarise]

def pipeline(doc):
    value = doc
    for stage in STAGES:
        value = stage(value)
        print("  after", stage.__name__ + ":", value)
    return value

pipeline("<invoice pdf bytes>")
#   after extract: {'amount': 420, 'currency': 'EUR', 'vendor': 'Acme'}
#   after validate: {'amount': 420, 'currency': 'EUR', 'vendor': 'Acme'}
#   after summarise: Acme invoiced 420 EUR

Parallel workers, results merged#

When the sub-tasks do not depend on each other, run them at once. This is the one case where splitting buys something a single agent cannot have at any price, because a single conversation is sequential by construction. Four questions that each take a fifth of a second take four fifths in series and one fifth in parallel.

import concurrent.futures, time

def worker(question):
    time.sleep(0.2)                       # stands in for a model call
    return question, "answer to " + question

questions = ["pricing", "uptime", "support hours", "data retention"]

start = time.time()
with concurrent.futures.ThreadPoolExecutor(max_workers=4) as pool:
    results = dict(pool.map(worker, questions))
print("parallel %.2fs" % (time.time() - start))       # parallel 0.20s

start = time.time()
results = dict(worker(q) for q in questions)
print("serial   %.2fs" % (time.time() - start))       # serial   0.81s

The merge is where this shape goes wrong. Collect results into a structure keyed by sub-task, never by appending to a shared list, and decide before you start what a missing result means: an empty entry the synthesis can see and mention, or a failed run. Silently merging three of four answers produces a confident summary of incomplete evidence, and there is nothing in the output that says so.

Shared state#

Agents that read and write shared state can overwrite each other or act on stale information. Keeping workers independent, each returning a result to the orchestrator rather than modifying shared data, avoids a whole class of concurrency bugs.

It is worth being precise about why this is worse here than in ordinary concurrent code. A thread that reads a stale value usually crashes or produces an obviously wrong number. An agent that reads a stale value writes a paragraph explaining it, and the paragraph is fluent. The lost update does not surface as an exception, it surfaces as a plausible answer nobody questions. If workers genuinely must share something, make it append-only, so the failure is a duplicate rather than a silent overwrite, and give every entry the id of the worker that wrote it so the trace can be read afterwards.

Two agents handing work back and forth#

A critic loop is a producer and a reviewer passing a draft between them. It is the pattern most likely to run away, because a reviewer asked to find problems will find problems, forever. The loop below has no termination condition, only a safety net, and the difference is the whole lesson: the net stops the bleeding, it does not make the loop correct.

class PingPong:
    def __init__(self):
        self.hops = 0
    def writer(self, draft):
        self.hops += 1
        return draft + " (revised)"
    def critic(self, draft):
        self.hops += 1
        return "needs work"               # a critic can always find something

pp = PingPong()
draft, verdict = "first draft", "needs work"
while verdict == "needs work":            # no termination condition
    draft = pp.writer(draft)
    verdict = pp.critic(draft)
    if pp.hops > 20:                      # only this line stops it
        print("stopped by the safety net after", pp.hops, "hops")
        break
# stopped by the safety net after 22 hops
print(len(draft), "characters")           # 121 characters

Bound it with three things at once, because each catches a different runaway. A step limit catches the loop that will not converge. A spend limit catches the loop that converges slowly on expensive calls. A deadline catches the worker that is not looping at all but waiting on something that will never arrive. Underneath all three there has to be a real termination condition, which is the only one of the four that ends the run because the work is done rather than because the run was cut off.

import time

class Budget:
    def __init__(self, max_steps, max_usd, deadline_s):
        self.max_steps, self.max_usd = max_steps, max_usd
        self.expires = time.monotonic() + deadline_s
        self.steps, self.usd = 0, 0.0
    def spend(self, in_tok, out_tok, price_in=1.0, price_out=5.0):
        self.steps += 1
        self.usd += in_tok / 1e6 * price_in + out_tok / 1e6 * price_out
    def exhausted(self):
        if self.steps >= self.max_steps:      return "step limit"
        if self.usd >= self.max_usd:          return "budget"
        if time.monotonic() >= self.expires:  return "deadline"
        return None

def critique_loop(budget, accept_after=3):
    rounds = 0
    while True:
        stop = budget.exhausted()
        if stop:
            return "stopped by " + stop + " after " + str(rounds) + " rounds, spent " + format(budget.usd, ".4f")
        budget.spend(3000, 800)               # writer
        budget.spend(3800, 200)               # critic
        rounds += 1
        if rounds >= accept_after:            # the real termination condition
            return "accepted after " + str(rounds) + " rounds, spent " + format(budget.usd, ".4f")

print(critique_loop(Budget(max_steps=20, max_usd=1.00, deadline_s=60)))
# accepted after 3 rounds, spent 0.0354
print(critique_loop(Budget(max_steps=4, max_usd=1.00, deadline_s=60)))
# stopped by step limit after 2 rounds, spent 0.0236
print(critique_loop(Budget(max_steps=20, max_usd=0.02, deadline_s=60), accept_after=99))
# stopped by budget after 2 rounds, spent 0.0236

Skip the shapes not listed here. Agents that vote, negotiate or hold debates are a research topic, and the honest version of every one of them in production is a supervisor that asks the same question twice and compares. If a design needs more than a supervisor, a pipeline and a parallel fan-out, the thing to suspect first is the decomposition, not the absence of a fourth pattern.

One practical note about combining them, since real systems always do. The composition that works is a pipeline on the outside with a fan-out in one stage, because then the stages are fixed and auditable and only the width of one stage varies. The composition that does not work is a supervisor whose workers are themselves supervisors. Two levels of dynamic planning means nothing in the run was decided by you, the trace has no stable shape to read, and the cost of a single request has no ceiling you can state. If you find yourself drawing that in an interview, the recovery is to say it out loud and flatten it.