Why Are We Still Governing AI Agents Like Software? - Tim Tyler, Gravitee
Loading slides…
Read the talk
Why Are We Still Governing AI Agents Like Software?
Tim Tyler of Gravitee develops an Agent Accountability framework around identity, a named human sponsor, bounded delegation, and continuous oversight. Its central distinction is practical: an agent can stay within its permissions and still fail at its job.
From a talk by Tim Tyler
At a glance
Ideas worth remembering
Give every agent a unique identity in a shared registry and a named human sponsor who owns its purpose, costs, and response when something goes wrong.
Separate the sponsor-defined standing role from the agent's current mission, and preserve permission limits across the full delegation chain rather than combining participants' access.
Measure both compliance and success. Invoice permissions can be checked directly, while the correctness of a purchase-order match needs a task-quality judgment with confidence attached.
Define recourse before failures occur: map uncertain evaluations and repeated rule breaches to appropriate actions, automate rapid enforcement, and keep the human sponsor responsible for revising the role and permissions.
The job description nobody would approve
A marketing-operations job requires no identification, has no manager, and offers only an obscure description of the work. The new hire gets access to key systems through shared passwords, works around the clock, receives no supervision, and faces no consequences for mistakes. Tim Tyler of Gravitee opens with this deliberately unreasonable vacancy. Nobody in the audience raises a hand to approve it. Yet these are the conditions he sees organizations creating for AI agents.
The organizational gap appears in a conversation Tyler recounts with an AI leader: the CTO does not quite consider agents software applications, while HR considers them precisely that. Responsibility falls between departments. Protocols, standards, and conventional software controls remain necessary; the proposed addition is a way to govern delegated work and identify who answers for it.
The Agent Accountability Framework borrows structures from managing a human workforce and adapts them to agents. The talk develops three stages: Assignment establishes the agent's place in the organization; Alignment defines its work and permitted scope; Administration checks outcomes and takes action when behavior departs from expectations. These are organizing principles backed by auditing controls, rather than a particular agent architecture.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Assignment: give the agent an identity and someone who answers
Assignment starts with an ID badge—ideally without the photograph that competes with your DMV picture for the worst of your life. The technical requirement beneath the joke is one unique agent ID in a known, shared registry that acts as the single source of truth. Every other part of the framework points back to that identity. Without it, the organization cannot consistently attach a role, permissions, or responsibility to the same agent.
The agent also belongs to a team, even when that team consists of just one human and one agent. Team membership supplies context: goals, previous decisions, and history. Information must travel back as well, so humans and other agents know what the agent is doing and learning. Calling it a teammate preserves an important distinction: acting on a person's behalf does not make the agent that person.
Agent 409 makes the responsibility concrete. Its sponsor is a named person, Sven. He defines its purpose, owns what it costs and spends, and gets the call when something goes wrong. That name connects the agent to the organization's existing management hierarchy. A team name alone would leave unanswered which person must respond.
Tyler's pool-safety rule explains what accepting sponsorship means. When he needed another adult to watch his children, he made eye contact and asked, “Do you have the kids?” The other adult had to answer, “I have the kids.” Responsibility transferred through an explicit acknowledgment. Sven's equivalent is “I have the agent”: ownership must be accepted, not merely assumed because several people are nearby.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Alignment: separate the standing role from the current mission
Once the organization knows who Agent 409 is and who answers for it, the next question is what it should do. A vague job description forces a human employee to infer expectations and check with colleagues. The framework instead gives the agent an explicit purpose, responsibilities for accomplishing it, a budget, and a definition of success. Those details provide something against which later work can be judged.
Two definitions operate at different timescales:
- Role at onboarding: Sven defines why the agent exists and its standing responsibilities. This changes rarely, unless subsequent intervention requires a revision.
- Mission at runtime: The agent defines what it is trying to accomplish at the current moment—for example, when making a call to an MCP tool.
The distinction keeps the agent's immediate choice of work separate from the sponsor's definition of its job. Defining a mission does not transfer ownership of the role to the agent.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Delegation must preserve the caller's scope
A role explains the job; authority determines which data and tools the agent can use to perform it. Sven decides which doors Agent 409 may open. IT, a database team, or an MCP-server team may implement the access, but the accountable sponsor decides that the agent needs it. This is also where limits and escalation conditions are defined.
What happens when the agent receiving a delegated task has more access than the agent making the request? Tyler uses three permission sets to make the relationship inspectable. Sven can access A, B, C, D. Agent 203, which discusses projects with him, can access C, D. Agent 409 can access D, E, F. Agent 203 delegates a task to Agent 409 on Sven's behalf.
In this example, Agent 409 may use D, E, F to do its work, but only D can return up the delegation chain. The common scope is {A, B, C, D} ∩ {C, D} ∩ {D, E, F} = {D}. Sven's access to A and B does not grant those permissions to the agents; Agent 409's access to E and F does not make that information available to Sven through Agent 203. The outward task delegation and the returning information therefore need separate treatment. The talk specifies the intended scope relationship, without detailing how a system would enforce it on derived results.
A delegated task does not combine everyone's permissions
Read the diagram as text
- Sven. Access: A, B, C, D
- Agent 203. Access: C, D. Discusses projects with Sven and delegates to Agent 409.
- Agent 409. Access: D, E, F. May use its own scope to perform the task, but may return only D through this chain.
- Sven → Agent 203: Task on Sven's behalf.
- Agent 203 → Agent 409: Delegated task.
- Agent 409 → Agent 203: Return limited to D.
- Agent 203 → Sven: Return limited to D.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The sponsor does not control every influence on the agent
“Never assume alignment” follows directly from the limits of sponsorship. Agent 409 inherits behavior from its underlying model, receives team context maintained elsewhere, and may interact with agents from other organizations. Sven cannot simply edit all of those influences. An outside agent should be treated like a contractor entering a building: give it an identity and authorization, including conditions on when it may access the organization, rather than assuming it shares the organization's intentions.
Sven can define Agent 409's own rules and intended outcomes. The rules provide a deterministic floor, such as fine-grained access policies. The intents describe what the agent should accomplish, where judging success may be probabilistic. This separates inherited alignment—which can change without Sven controlling it—from the defined expectations he can impose on Agent 409. Administration is the next stage because writing those expectations does not enforce them.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Administration: behaving is different from succeeding
An agent can stay within every access restriction and still do poor work. Authority defines acceptable behavior; the role defines success. Oversight must check both. That is why the framework calls for a scorecard rather than one metric: permission compliance cannot stand in for task quality, and a useful result cannot excuse unauthorized access.
What would the scorecard distinguish in invoice work?
- Authority checks: An illustrative agent may handle invoices only up to $40,000. Other example rules prohibit approving payments or permit writes only to the ledger. These are explicit constraints to check.
- Role outcomes: Invoice-match accuracy and reconciliation cycle time measure whether the work is useful. Matching the correct purchase order, vendor, and project requires judging the result against the intended task, with a confidence level attached to that judgment.
The role and authority definitions supply the scorecard's weights. As Tyler puts it, “rules are checked, intents are judged.” The distinction makes a compliant but incorrect match visible as a failure rather than a successful authorized action.
Oversight also needs to inspect more than the final answer. The proposed guardian agents watch the model, the context entering the agent, and the work product coming out. Those are separate places where behavior can diverge from expectations. Continuous checks at machine speed are the point of the otherwise unwelcome management analogy: agents need micromanagers. A quarterly review can arrive after enormous numbers of decisions have already happened.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn uncertain judgments into defined actions
Detecting a problem is only useful if the organization has decided what happens next. The human-workforce analogy starts with correction rather than immediate dismissal: explain the misunderstanding, restrict authority if problems continue, escalate, and eventually stop the work. Agent recourse needs similarly explicit responses, with different treatment for uncertain task judgments and clear permission violations.
How can a probabilistic evaluation trigger a predictable response? Tyler sketches confidence-linked actions: at 90% confidence, the system might redact out-of-alignment content; below 50%, it escalates to a human; below 20%, it blocks. The mechanism is to map an uncertain assessment to a predefined action. These are illustrative thresholds, not a complete policy: the talk does not define how confidence is calibrated or resolve the overlap between the below-50% and below-20% conditions.
An explicit access violation offers a different basis for action. Authorization can be restricted immediately after an unauthorized access; access might be suspended after five violations in a day; persistent violations over several days might suspend the entire agent. Here the response depends on a known rule breach and its repetition, rather than confidence in a semantic judgment. The escalation policy makes both the current incident and the agent's history consequential.
Tyler's proposed zero-trust posture begins with heavier controls and lightens them after repeated success. That creates an ongoing operational decision, not a setting to configure once and forget. The practical requirement is fast enforcement combined with continued examination of whether the restrictions and outcomes remain appropriate.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Machines enforce; the sponsor changes the job
Guardian agents, AI gateways, and guardrails are proposed enforcement mechanisms: they restrict, prohibit, or suspend activity at machine speed. Sven has a different job. He uses human judgment to ask whether changing the role or authority definition would produce better work or better behavior. Enforcement acts on the current expectations; coaching revises those expectations when they are causing the wrong result.
The closing formulation is compact: name every agent, bound every action, trace every outcome. The framework supplies principles and auditing controls; technical systems provide ways to carry them out. The first practical move is to give agents their own identities, so the organization can attach the rest of that responsibility to something identifiable. And, in the spirit of every employee badge Tyler has disliked, give the agent the worst picture of its life.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Use the framework's six questions—identity, sponsor, role, authority, oversight, and recourse—to examine an individual agent.
Related talks
- How to Know When an AI Agent Can Work Unsupervised — Evan Kormos, Coinbase
Tyler’s scorecard raises a practical question for anyone deciding how much supervision an agent needs. What evidence would justify lighter oversight?
- The Next Trillion Internet Users Still Don’t Have an Identity — Adi Singh, AgentMail
Tyler’s registry example shows why agent identity matters even within one organization. A unique ID ties an agent’s permissions and spending to a named sponsor.
Full transcript
Loading transcript…