Keeping AI agents under control: a method for predictable agent work
AI agents stay under control when people decide three things in advance and the system enforces them: what work may start, what it may cost, and what may go live. Give each task clear criteria and one budget, set money aside before a run, stop on repetition, keep the agent that writes apart from the code that publishes, and put a person at the gates that matter.
This guide describes the method we use to build our own product with AI agents, described in the agent factory case study. Every number in it is hypothetical, invented to show the arithmetic.
Decide who starts the work
An agent that can start its own work will find plenty: a failing test, a TODO in the code, an improvement it thinks would help. Each one may be reasonable. Together they are a team member nobody manages.
So the first rule is simple: work starts only when a person asks for it. In practice that means one explicit, recorded act by someone with the authority to approve it, such as a label on a ticket, a button, or a signed-off request.
| Possible trigger | Allowed to start work? | Why |
|---|---|---|
| A person approves a written task | Yes | Someone decided, and it is on record |
| An agent notices a failing test | No, it may propose a task | Proposals are cheap; unreviewed work is not |
| An agent has an idea | No, it may propose a task | Same |
| A scheduled job | Only if a person approved the schedule | The decision moved earlier; it did not disappear |
Let agents propose as much as they like. Proposals land in a queue a person reads.
Read the diagram as text
- Person: Approves a written task
- System: Opens one budget for the task
- Agent: Works inside its scope (named files, its own copy, no secrets)
- Code: Checks the change (scope, secrets, protected files). Otherwise: Refused: a person decides
- Agent: Tests and reviews (against the written criteria)
- Person: Decides what goes live
Write tasks a machine can check
Most wasted agent runs start from a task that could not be done as written. A file that does not exist, two requirements that contradict each other, no way of telling when the job is finished.
A task an agent can work from has four parts:
- What to change, in a sentence or two.
- Where: the files, folders or systems it may touch, and by implication everything it may not.
- How to check it: numbered acceptance criteria that a test or a reviewer can confirm one by one.
- What not to do, when there is an obvious wrong path.
Then put a cheap check in front of the expensive work. Before an agent spends real effort, a quick, low-cost read of the task asks: is this workable? Are the files there? Are the criteria testable? Refuse the task at the door if not. Refuse two tasks whose scopes overlap, too, so agents working in parallel never edit the same files.
Budget the task, and set money aside before spending
A limit per model call protects you from one expensive answer. It does nothing about a task that makes many cheap calls, stops, restarts and makes many more. What you want to control is what a piece of work costs from start to finish.
Read the diagram as text
- Ring 1: Monthly limit: what the whole system may spend in a month
- Ring 2: Daily limit: so one bad day cannot use the month
- Ring 3: Task budget: shared by every agent that touches the task
- Ring 4: Money set aside for a run: held before the run starts, settled when it ends
- At the centre: The agent at work
Agents run in parallel, so count spend before a run starts, not after it ends. Each run sets its own limit aside in one step; when it ends, the amount held is replaced by what it actually cost.
room for a new run = daily limit - spent today - (live runs × amount held per run)
start a run only if room for a new run >= amount held per run
Two more habits make the budget hold:
- Enforce turn and time limits in your own code. A model will not always stop exactly at the number you gave it. Your code should.
- Let started work finish. A daily limit that stops everything at once freezes tasks one step from done. Keep a small allowance that only work already started may use.
| Level | What it limits | Who changes it |
|---|---|---|
| Run | One attempt by one agent | Set in code, changed through review |
| Task | Everything done for one request | Set in code, changed through review |
| Day | All tasks together | A person, with a written reason |
| Month | All tasks together | The budget owner |
Cap attempts and stop on repetition
Money is not the only thing that runs away. A cheap agent that keeps retrying burns time, fills the board with noise and can still stay under budget. Give each kind of agent a number of attempts per task as well.
The most useful single rule we know: if an agent reports the same problem twice, stop. Nothing about the third attempt will be different, and paying for it three times is how small costs become large ones.
Checklist for a repetition stop that works:
- Record what each run observed in a normalised form (the kind of error and where, not the full text), so two identical stops compare equal.
- Compare with the previous observation for the same task, not across tasks.
- Stop before starting the next attempt, not after it.
- Send a stopped task first to an agent that can clarify the task, and only then to a person, with both logs attached.
Separate the agent that writes from the code that publishes
If the agent that writes a change can also publish it, every check you add is advice. Split the two.
| The agent that writes | The code that publishes |
|---|---|
| Works in its own copy of the code | Reads every change before anything leaves |
| Has no permission to publish | Holds the only permission to publish |
| Cannot read secrets | Refuses anything shaped like a credential |
| Stays within the files the task named | Flags or refuses changes outside that scope |
| Cannot change the safety rules | Refuses changes to its own rules unless a person allowed it |
Make the publishing step plain code, not another model. It should be boring, predictable and easy to test. Decide which problems block a change and which only flag it for a person: refusing to publish leaves the work harder to review, not safer, so reserve refusal for what truly must not go out.
Keep a person at the gates that matter
Agents can do most of the work between the start and the end. The start and the end belong to people.
- At the start, a person decides the work is worth doing and writes how to check it.
- At the end, a person reviews, or a written rule a person owns decides which kinds of change may go live on their own.
- Every change reports what it did not verify. That one habit tells the reviewer where to look.
Widen automatic approval slowly, one kind of change at a time, each as a written decision with a named owner. Trust is earned per category, not granted to the agent as a whole.
Learn from every run
After each task, ask one question: what about the system made this harder, and what one change would stop it? A short, cheap retrospective can answer it from the run’s own log.
Two rules keep this useful. It proposes and never changes anything itself; a person decides what becomes work. And an empty answer is a good answer: a retrospective that always finds something is producing noise.
| After every task | Who | Output |
|---|---|---|
| Read the log, propose up to a few changes to the system | A small agent | Proposals in a queue |
| Review the queue | A person | Accepted, merged with similar ones, or dropped |
| Build accepted changes, each with a test that fails first | The system, through the normal process | A better system for the next task |
Work through a hypothetical example
Every number in this section is invented. It shows how the rules combine, not what any real day costs.
Invented inputs
| Input | Hypothetical value |
|---|---|
| Daily limit | 100 |
| Spent today by finished runs | 52 |
| Live runs | 3 |
| Amount held per run | 12 |
| Allowance for started work only | 20 |
| Task budget | 15 |
Calculate the room for new work
held by live runs = 3 × 12 = 36
room for a new run = 100 - 52 - 36 = 12
new run? 12 >= 12 -> yes, exactly one
after it starts = 12 - 12 = 0 -> no new work until a run finishes
started work = 0 + 20 = 20 >= 12 -> a task already under way may take its next step
Read the diagram as text
- Spent by finished runs: 52
- Held by 3 live runs: 36
- Room for a new run: 12
- Allowance, started work only: 20
- Ceiling: Daily limit 100
Now suppose one of the live tasks has already used 11 of its 15. Its next run would need 12 set aside, which the task budget cannot cover, so under these rules the task stops and goes to a person, even though the day still has room. That is the task budget doing its job.
If you want AI agents in your team and need their work to stay safe and predictable, book a call. We will go through your setup with you and tell you where the controls should sit.
Frequently asked questions
What is the biggest risk when a team starts using AI agents?
Not a single bad answer, but work nobody decided on, spend nobody noticed and changes nobody checked. Each of these is cheap to prevent up front and expensive to discover afterwards. The method in this guide closes all three before the agent starts.
Where should the budget sit, on each call, each task or each month?
All three exist, but the one that decides most often is the budget for each task. A per-call limit cannot see a task that loops through many cheap calls, and a monthly cap only tells you at the end. A budget for each task, shared by every agent that touches it, stops a runaway task while it is still small.
Why set money aside before a run instead of counting it afterwards?
Because agents run in parallel. If several check the remaining budget at once, each sees enough and each starts, and together they overspend. Setting the run's limit aside in one step before it starts makes the check and the spend the same operation.
Do we still need people if the agents are good?
Yes, at the start and at the end. A person decides what work is worth doing and writes down how to check it, and a person, or a written rule a person owns, decides what goes live. In between, agents and plain code can do most of the work.
Does this depend on a particular model or tool?
No. The principles work with any agent your own code can start, stop and inspect. The tools change every few months; who decides what, and what is enforced before money is spent, should not.
How long does it take to set up?
It depends on how many systems the agents will touch and how much of your process is already written down, so we do not quote a number before looking. The fastest start is one team and one kind of well-defined task, which shows within weeks whether the approach suits you.
Where to start
Pick one kind of task your team does often and could describe in a few lines: what to change, where, and how to check it. Write five of them down as if for a new colleague. If that is easy, the work is ready for an agent and the rest of this guide tells you how to keep it safe. If it is hard, the first win is clearer tasks, for people as much as for agents.