Defending architecturally

If injection cannot be prevented, the question becomes what happens when it succeeds. This section is about designing the agent so the answer is: not much. The controls are ordinary security ones applied to a component you cannot trust — give each tool the narrowest permission that still works, put a person in front of anything irreversible, keep untrusted text away from privileged actions, and close the routes data could leave by. Skip it and one poisoned document is an exfiltration incident rather than an odd line in a log.

Assume the injection succeeds#

Design as though the model will eventually be persuaded to do the wrong thing, because sooner or later it will. The question is what it can do at that moment.

Least privilege#

The most effective control is not giving the tool the capability in the first place. An agent that can read orders cannot delete them regardless of what it is told.

# Too broad: one compromised turn can do anything
def run_sql(query: str) -> list[dict]:
    return db.execute(query)

# Scoped: the blast radius is the signature
def find_orders(email: str, limit: int = 10) -> list[dict]:
    return db.orders(email=email, limit=min(limit, 10))

Gate the consequential actions#

Anything irreversible or externally visible should require a person. Sending money, deleting data, emailing a customer, publishing. The agent proposes; a human confirms.

@requires_approval
def issue_refund(order_id: str, amount: float):
    ...

# The agent's call returns a pending action rather than doing it,
# and a person sees exactly what is proposed before it runs.

Separate trust levels#

Do not let a single agent both read untrusted content and hold dangerous permissions. Split it: one component reads and extracts, with no tools; another acts, on structured input only.

Untrusted document
      |
      v
  Extractor        no tools, output constrained to a schema
      |
      v
  Structured data  no free text carried forward
      |
      v
  Actor            has tools, never sees the raw document

Contain exfiltration#

• Do not let the agent fetch arbitrary URLs. A URL is an excellent way to carry data out.
• Allowlist domains for any fetching tool rather than blocklisting.
• Be careful with rendered markdown images in output, since an image URL is a silent outbound request.
• Log what the agent read and what it sent, so an incident can be reconstructed.