AI agent governance: the decisions to make before an agent touches your systems

In short. AI agent governance is the small set of decisions that say what an agent may do, who owns it, and what gets written down while it works. Six cover most of it: ownership, what the agent may reach, what is recorded, who may change it, what happens when it fails, and when it is switched off. Make them before the first agent runs. After that, each one is a change request.

Many teams write an AI policy first. The policy says the organisation will use AI responsibly, everyone signs it, and six weeks later an agent is working in production and nobody can say who approved the systems it reaches. A policy states an intention. Governance is the duller set of decisions that a person can check afterwards.

An agent makes this harder than ordinary software, for one reason. A script does the same thing every time, so reading the code tells you what it will do. An agent picks its own steps while the job runs. So the useful question is not "what will it do", it is "what is it able to do, and what did it do last Tuesday".

The six decisions below are the ones a risk owner or a procurement lead will ask about, and each is cheaper to make before the first agent runs.

One named person per agent, written down. Not a team, not a department: a person who can answer what the agent is for, what it may touch, and who asked for it. Ownership sounds like paperwork until the first odd result appears and three teams each believe another one is watching.

Give the owner two duties that take minutes: confirm every quarter that the agent is still needed, and that its tool list is still right. An agent nobody has confirmed for a year is a finding waiting to happen.

List the tools the agent may use, and the limit on each one. Tools are how an agent changes anything. Without them it can only produce text.

Start from nothing allowed, and add what the job needs. For each tool, write down the narrow version: which folder, which account, which records, read-only or not. "Access to the document store" is not a decision; "read access to the tender folder for this year" is. The expensive mistake is rarely the model writing a wrong sentence. It is a tool doing something real.

Decide before you start what a run has to leave behind: the steps, the inputs, the outputs, the times, and which tool was called with what. The model's own summary of its work is not the record. A model writes text that reads like a report, and that text comes from the same system you are trying to check.

Two further decisions belong here. How long you keep the record, which follows your own retention rules and not the vendor's default. And who may read it, because a record of every run is also a record of the people whose cases the agent handled.

An agent has more moving parts than a script: the instructions it is given, the model behind it, the tool list, and the documents it can search. Any of the four changes its behaviour, and three of them can be changed by someone who is not a developer.

So name who may change each part, and decide what counts as an approved change. Keep the old version. A team that cannot say which version of an agent produced last month's output is going to have a bad afternoon explaining it.

Write the answer down while everyone is calm. Who can stop the agent, and how fast. What is undone, and by whom. Who is told, inside the organisation and outside it. Where the evidence is kept while the incident is open. This is the part teams skip because it feels unlikely, and it is the part that turns a bad hour into a bad quarter.

Every agent gets an end date or a review date. Say what happens to the record, the indexed documents and the credentials when it stops. Organisations are good at starting pilots and poor at ending them, and a forgotten agent keeps its access long after its project has closed.

One pattern is worth naming, because it defeats good intentions. Teams start with a person reading every result. The agent is right most of the time, reading gets dull, and the reading stops. Nothing fails and nothing warns you. On paper the control is still there; in practice it went away weeks ago.

So prefer controls that are part of how the system runs, not controls that depend on somebody staying alert. A short tool list and a complete record are still there on a busy Friday.

Governance is your decision set, but the platform either supports it or fights it. Three things decide which:

• A place to see an agent whole — its instructions, its tools and its knowledge in one view, so one person can review it before it goes live.
• A record the runtime writes — the steps, the inputs, the tool calls and the results, in a form you can export into your own systems.
• A licence that lets your own people read the code. The licence file is the only place that answers this. Read it before the demo, not after.

If one of these is missing, your team builds it later, usually after the first incident.

The deployment question travels with the governance question: whose infrastructure the agent runs on, which model it calls, and who holds the record when someone asks what happened. If the record sits in a vendor's cloud, your account of last Tuesday depends on that vendor's cooperation and retention settings. If you run the agent in your own environment, the six decisions above are yours to make and yours to prove. We looked at the layer that does the recording in who audits the harness.