Why Are We Still Governing AI Agents Like Software? - Tim Tyler, Gravitee

Loading slides…

Read the talk

Why Are We Still Governing AI Agents Like Software?

Tim Tyler of Gravitee develops an Agent Accountability framework around identity, a named human sponsor, bounded delegation, and continuous oversight. Its central distinction is practical: an agent can stay within its permissions and still fail at its job.

From a talk by Tim Tyler

At a glance

Ideas worth remembering

  • Give every agent a unique identity in a shared registry and a named human sponsor who owns its purpose, costs, and response when something goes wrong.

  • Separate the sponsor-defined standing role from the agent's current mission, and preserve permission limits across the full delegation chain rather than combining participants' access.

  • Measure both compliance and success. Invoice permissions can be checked directly, while the correctness of a purchase-order match needs a task-quality judgment with confidence attached.

  • Define recourse before failures occur: map uncertain evaluations and repeated rule breaches to appropriate actions, automate rapid enforcement, and keep the human sponsor responsible for revising the role and permissions.

The job description nobody would approve

A marketing-operations job requires no identification, has no manager, and offers only an obscure description of the work. The new hire gets access to key systems through shared passwords, works around the clock, receives no supervision, and faces no consequences for mistakes. Tim Tyler of Gravitee opens with this deliberately unreasonable vacancy. Nobody in the audience raises a hand to approve it. Yet these are the conditions he sees organizations creating for AI agents.

The organizational gap appears in a conversation Tyler recounts with an AI leader: the CTO does not quite consider agents software applications, while HR considers them precisely that. Responsibility falls between departments. Protocols, standards, and conventional software controls remain necessary; the proposed addition is a way to govern delegated work and identify who answers for it.

The Agent Accountability Framework borrows structures from managing a human workforce and adapts them to agents. The talk develops three stages: Assignment establishes the agent's place in the organization; Alignment defines its work and permitted scope; Administration checks outcomes and takes action when behavior departs from expectations. These are organizing principles backed by auditing controls, rather than a particular agent architecture.

0:130:42
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:13 · section reference included

Assignment: give the agent an identity and someone who answers

Assignment starts with an ID badge—ideally without the photograph that competes with your DMV picture for the worst of your life. The technical requirement beneath the joke is one unique agent ID in a known, shared registry that acts as the single source of truth. Every other part of the framework points back to that identity. Without it, the organization cannot consistently attach a role, permissions, or responsibility to the same agent.

The agent also belongs to a team, even when that team consists of just one human and one agent. Team membership supplies context: goals, previous decisions, and history. Information must travel back as well, so humans and other agents know what the agent is doing and learning. Calling it a teammate preserves an important distinction: acting on a person's behalf does not make the agent that person.

Agent 409 makes the responsibility concrete. Its sponsor is a named person, Sven. He defines its purpose, owns what it costs and spends, and gets the call when something goes wrong. That name connects the agent to the organization's existing management hierarchy. A team name alone would leave unanswered which person must respond.

An illustrated sponsor hands a badge to an agent beside three cards: Defines Agent 409; Owns what 409 spends; and Gets the call if something goes wrong. A statement below says that one name makes the agent’s line of accountability clear.Open full source frame
Sven sponsors Agent 409. He defines its job, owns its spending and gets the call when something goes wrong.

Tyler's pool-safety rule explains what accepting sponsorship means. When he needed another adult to watch his children, he made eye contact and asked, “Do you have the kids?” The other adult had to answer, “I have the kids.” Responsibility transferred through an explicit acknowledgment. Sven's equivalent is “I have the agent”: ownership must be accepted, not merely assumed because several people are nearby.

2:423:12
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:42 · section reference included

Alignment: separate the standing role from the current mission

Once the organization knows who Agent 409 is and who answers for it, the next question is what it should do. A vague job description forces a human employee to infer expectations and check with colleagues. The framework instead gives the agent an explicit purpose, responsibilities for accomplishing it, a budget, and a definition of success. Those details provide something against which later work can be judged.

Two definitions operate at different timescales:

  • Role at onboarding: Sven defines why the agent exists and its standing responsibilities. This changes rarely, unless subsequent intervention requires a revision.
  • Mission at runtime: The agent defines what it is trying to accomplish at the current moment—for example, when making a call to an MCP tool.

The distinction keeps the agent's immediate choice of work separate from the sponsor's definition of its job. Defining a mission does not transfer ownership of the role to the agent.

Slide headed “Onboarding vs. runtime scope.” A Purpose card says “Match invoices to purchase orders and post the entries” and “Set once. Changes rarely.” An illustrated octopus agent wearing badge 409 appears beside the card.Open full source frame
Agent 409’s standing job is to match invoices to purchase orders and post the entries. That purpose is set at onboarding.
5:426:12
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:42 · section reference included

Delegation must preserve the caller's scope

A role explains the job; authority determines which data and tools the agent can use to perform it. Sven decides which doors Agent 409 may open. IT, a database team, or an MCP-server team may implement the access, but the accountable sponsor decides that the agent needs it. This is also where limits and escalation conditions are defined.

What happens when the agent receiving a delegated task has more access than the agent making the request? Tyler uses three permission sets to make the relationship inspectable. Sven can access A, B, C, D. Agent 203, which discusses projects with him, can access C, D. Agent 409 can access D, E, F. Agent 203 delegates a task to Agent 409 on Sven's behalf.

In this example, Agent 409 may use D, E, F to do its work, but only D can return up the delegation chain. The common scope is {A, B, C, D} ∩ {C, D} ∩ {D, E, F} = {D}. Sven's access to A and B does not grant those permissions to the agents; Agent 409's access to E and F does not make that information available to Sven through Agent 203. The outward task delegation and the returning information therefore need separate treatment. The talk specifies the intended scope relationship, without detailing how a system would enforce it on derived results.

Slide states “Never more than the lowest authority scope in the lineage.” Orange arrows connect Sven with gray A–D tiles to Agent 203 with orange C and D tiles, then to Agent 409 with an orange D tile and gray E and F tiles.Open full source frame
Follow permission D from Sven through Agent 203 to Agent 409. It is the only permission they all share.

A delegated task does not combine everyone's permissions

SvenAccess: A, B, C, D
Agent 203Access: C, D. Discusses projects with Sven and delegates to Agent 409.
Agent 409Access: D, E, F. May use its own scope to perform the task, but may return only D through this chain.
Task delegation moves toward Agent 409. Returning information is limited to D, the only scope shared by every participant; Agent 409's local access to E and F does not expand the upstream caller's access.
Read the diagram as text
  • Sven. Access: A, B, C, D
  • Agent 203. Access: C, D. Discusses projects with Sven and delegates to Agent 409.
  • Agent 409. Access: D, E, F. May use its own scope to perform the task, but may return only D through this chain.
  • Sven → Agent 203: Task on Sven's behalf.
  • Agent 203 → Agent 409: Delegated task.
  • Agent 409 → Agent 203: Return limited to D.
  • Agent 203 → Sven: Return limited to D.
7:328:02
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:32 · section reference included

The sponsor does not control every influence on the agent

“Never assume alignment” follows directly from the limits of sponsorship. Agent 409 inherits behavior from its underlying model, receives team context maintained elsewhere, and may interact with agents from other organizations. Sven cannot simply edit all of those influences. An outside agent should be treated like a contractor entering a building: give it an identity and authorization, including conditions on when it may access the organization, rather than assuming it shares the organization's intentions.

Sven can define Agent 409's own rules and intended outcomes. The rules provide a deterministic floor, such as fine-grained access policies. The intents describe what the agent should accomplish, where judging success may be probabilistic. This separates inherited alignment—which can change without Sven controlling it—from the defined expectations he can impose on Agent 409. Administration is the next stage because writing those expectations does not enforce them.

9:3210:01
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:32 · section reference included

Administration: behaving is different from succeeding

An agent can stay within every access restriction and still do poor work. Authority defines acceptable behavior; the role defines success. Oversight must check both. That is why the framework calls for a scorecard rather than one metric: permission compliance cannot stand in for task quality, and a useful result cannot excuse unauthorized access.

What would the scorecard distinguish in invoice work?

  • Authority checks: An illustrative agent may handle invoices only up to $40,000. Other example rules prohibit approving payments or permit writes only to the ledger. These are explicit constraints to check.
  • Role outcomes: Invoice-match accuracy and reconciliation cycle time measure whether the work is useful. Matching the correct purchase order, vendor, and project requires judging the result against the intended task, with a confidence level attached to that judgment.

The role and authority definitions supply the scorecard's weights. As Tyler puts it, “rules are checked, intents are judged.” The distinction makes a compliant but incorrect match visible as a failure rather than a successful authorized action.

Oversight also needs to inspect more than the final answer. The proposed guardian agents watch the model, the context entering the agent, and the work product coming out. Those are separate places where behavior can diverge from expectations. Continuous checks at machine speed are the point of the otherwise unwelcome management analogy: agents need micromanagers. A quarterly review can arrive after enormous numbers of decisions have already happened.

11:0211:32
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:02 · section reference included

Turn uncertain judgments into defined actions

Detecting a problem is only useful if the organization has decided what happens next. The human-workforce analogy starts with correction rather than immediate dismissal: explain the misunderstanding, restrict authority if problems continue, escalate, and eventually stop the work. Agent recourse needs similarly explicit responses, with different treatment for uncertain task judgments and clear permission violations.

How can a probabilistic evaluation trigger a predictable response? Tyler sketches confidence-linked actions: at 90% confidence, the system might redact out-of-alignment content; below 50%, it escalates to a human; below 20%, it blocks. The mechanism is to map an uncertain assessment to a predefined action. These are illustrative thresholds, not a complete policy: the talk does not define how confidence is calibrated or resolve the overlap between the below-50% and below-20% conditions.

An explicit access violation offers a different basis for action. Authorization can be restricted immediately after an unauthorized access; access might be suspended after five violations in a day; persistent violations over several days might suspend the entire agent. Here the response depends on a known rule breach and its repetition, rather than confidence in a semantic judgment. The escalation policy makes both the current incident and the agent's history consequential.

Tyler's proposed zero-trust posture begins with heavier controls and lightens them after repeated success. That creates an ongoing operational decision, not a setting to configure once and forget. The practical requirement is fast enforcement combined with continued examination of whether the restrictions and outcomes remain appropriate.

14:0114:32
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:01 · section reference included

Machines enforce; the sponsor changes the job

Guardian agents, AI gateways, and guardrails are proposed enforcement mechanisms: they restrict, prohibit, or suspend activity at machine speed. Sven has a different job. He uses human judgment to ask whether changing the role or authority definition would produce better work or better behavior. Enforcement acts on the current expectations; coaching revises those expectations when they are causing the wrong result.

The closing formulation is compact: name every agent, bound every action, trace every outcome. The framework supplies principles and auditing controls; technical systems provide ways to carry them out. The first practical move is to give agents their own identities, so the organization can attach the rest of that responsibility to something identifiable. And, in the spirit of every employee badge Tyler has disliked, give the agent the worst picture of its life.

16:0216:32
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:02 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…