Roll out an AI agent at work in safe stages

---
name: "roll-out-an-ai-agent-in-stages-method"
description: "Use when you need to Roll out an AI agent at work in safe stages. Pick one job, build a test set, run the agent in shadow mode, add guardrails, then widen it in steps"
---

# Pick one job, build a test set, run the agent in shadow mode, add guardrails, then widen it in steps

Job: [Roll out an AI agent at work in safe stages](https://aiagents.org/canai/roll-out-an-ai-agent-in-stages)

## Need

- AI agent

## Steps

1. Pick a job, not a technology. Start with one workflow that has volume, a clear definition of done and an owner. Write down today’s cost per task: time, error rate and who fixes the errors. Try the simplest design that could work first: one model call, then a fixed sequence of steps, and only then an agent that chooses its own steps. Working when: One page states the task, the baseline and what good looks like.
2. Build the test set before the agent. Collect 20 to 50 real cases with the outcome you expect, starting with the ones that went wrong before. Grade automatically where an answer can be checked, and have a person grade the rest. Rerun the full set after every prompt, tool or model change. Working when: A change can be tested in minutes, not argued about in a meeting.
3. Run it in shadow mode. The agent drafts and people decide. Nothing it does reaches a customer or a system of record yet. Compare its output with what the person did, on real work, for at least two weeks. Log every model call and tool call, so you can see why it went wrong. Working when: You know its agreement rate on real traffic, not on a demo.
4. Add guardrails and handoffs. Decide in advance what the agent may do alone, what needs approval and when it stops. Give it the fewest permissions that do the job, and require a person to approve anything irreversible: payments, deletions, outside email. Hand off to a person after repeated failures or on high-risk requests, and give each agent its own budget at the gateway. Working when: Every action the agent can take is on a list with an owner.
5. Scale on evidence. Widen in steps, such as 10%, then 50%, then all of the work, and only while the numbers hold. Review cost per task, error rate and handoffs every month against the baseline from stage 1. Reuse the gateway, test harness and logs for the second workflow. That is where the next one gets cheaper. Working when: Cost per task and error rate both beat the baseline.
6. Note: Adapted (formatting only) from Steel Town AI's Agent Rollout Ladder (CC BY 4.0). Industry figures on the source page are left out.

## Credit

Method author Steel Town AI (CC BY 4.0)
Source: https://github.com/steeltownai/toolkit/blob/main/frameworks/agent-rollout-ladder.md