What is AI governance?

Governance is a Capability, not a Document

The most useful shift in thinking about AI governance is to stop asking whether a governance document exists and start asking what the organisation can answer.

This is how scrutiny actually arrives. When an incident happens, nobody asks whether you had a governance framework. A board, a regulator, or a client asks you to explain what happened — and explaining what happened means producing evidence about a specific decision, not pointing at a policy that covered agents in general. Governance defined as an artefact (a policy, a committee, a risk register) and governance defined as a capability (can you answer the question?) are different things, and only the second one helps when the question lands.

So a working definition: AI governance is the set of mechanisms that determine, constrain, and account for what an AI system does — tested not by whether the rules are written down, but by whether the organisation can answer for a specific decision after the fact.

The Four Questions a Governed System Can Answer

A governed AI system can answer four questions about a specific decision, with evidence, after it was made. They are worth stating plainly because they double as a diagnostic — an organisation can check itself against them directly.

What was this system permitted to do? Not "what are our AI policies" but what this agent, in this deployment, was actually allowed to do, and where that was recorded. A strong acceptable-use policy answers the general question and not the specific one.

What was it working from when it acted? The goals it was given, the constraints it was under, the operating context it drew on. This is distinct from the prompt: the prompt is what was typed, the question is the full context the system actually worked from. Most architectures don't preserve this as a retrievable record.

What did it actually do? Logs, traces, outputs. This is the question the monitoring industry has largely solved. It tells you what happened — though not whether what happened was reasonable given what the system had to work with.

Who is accountable for it? Agents are not legal persons. Accountability sits with the organisation and has to land on a specific role. Diffuse responsibility functions as no accountability at all when a decision needs explaining.

Where the Gap is

These four questions map onto the lifecycle of a decision — what the system was permitted and given before it acted, what it did during, what can be reviewed after. That before/during/after shape is the familiar one in governance frameworks. The useful observation is where the four questions fall on it.

Questions three and four are widely addressed. Question one is partially addressed. Question two — what the system was working from — is the one most deployments cannot answer, and it is not a minor omission. A decision can't be judged reasonable or unreasonable without it, because reasonableness is always relative to what the system had to work with.

The asymmetry is concrete, not theoretical. Many governance programmes can produce their approval workflows and their monitoring dashboards on request, yet cannot retrieve the exact operating context an agent used for a decision made six months earlier. The first two were recorded by design. The third was never captured as a durable record. That is the shape of the gap in most current deployments — and it sits precisely between the rules and the observation, in the layer neither governance policy nor monitoring is built to hold.