All resources
Agents15 June 2026·6 min read

Why your first agent should fail safely: designing the blast radius before the brain

Teams debate which model to use for their first agent. The question that actually determines whether it survives contact with production is: what's the worst thing it can do?

Ask a team building their first agent which model they're using, and they'll answer immediately. Ask what the agent's worst plausible action is, and the room goes quiet. That's backwards. Model choice is a decision you can revisit next month; an agent that already sent the wrong refund, deleted the wrong record, or emailed the wrong customer is a decision you can't take back. Blast radius — the scope of damage a single wrong action can cause — should be designed before a single prompt is written.

The design questions that come before the model

  • What accounts can this agent act as? A service account with admin-equivalent access turns a small reasoning error into a large operational one — scope credentials to the narrowest role the task requires.
  • What actions are reversible? Draft-and-send should not share a permission tier. Anything irreversible (a payment, a deletion, an external communication) gets a human gate regardless of the agent's confidence.
  • What's the quarantine path? When the agent behaves unexpectedly, is there a kill switch that stops it mid-run without taking down the whole system, and a way to review exactly what it did before it's allowed to resume?
  • What's the maximum single-action impact? Rate limits and batch caps on agent actions turn a bug that would otherwise touch ten thousand records into one that touches ten, caught long before it compounds.

A useful design rule: if you can't describe the worst thing your agent could plausibly do in one sentence, you haven't scoped it — you've hoped at it.

This is a cheaper conversation to have early

Blast-radius design costs an afternoon of workshop time before any code exists. Retrofitting it after an incident costs an incident review, a trust repair with whichever team got burned, and usually a much more conservative rollout of every agent project that follows. The teams that ship agents fastest, longer term, are the ones who were slowest and most paranoid about this one design pass.

Related capability

This is what our Autonomous AI Agents practice is built around.

See how it's scoped