[
  {
    "startMs": 12840,
    "text": "My name is Kim Maida. I work at Keycard, and Keycard provides agent access, identity, and access at runtime for agents. Oh, my."
  },
  {
    "startMs": 25960,
    "text": "Hi, everyone. My name is Mason Egger. I'm a solutions architect at Temporal. And, uh, Temporal provides durable execution to your software applications up into and including agents. So not just agents, but pretty much the entire bu- the entire stack. So."
  },
  {
    "startMs": 43560,
    "text": "So everything you're gonna need for the workshop is at the link that's on the screen right now. You actually do not need to sign up for Keycard ahead of time because the sign-up will be included, like, within the curriculum. But in order to get an account for your own personal use, if you want to play with Keycard on your own after the workshop, you can go to the link that is provided at the bottom and enter that registration code in order to sign up for a personal account."
  },
  {
    "startMs": 76200,
    "text": "Okay. So we'll come back to this slide, but I am going to start out by talking a little bit about agent access."
  },
  {
    "startMs": 86400,
    "text": "So one of the things that we do a lot of these days is give API keys to agents to give them access to resources, right? And there's several problems with doing things this way. One being credentials aren't attributed to a user identity or an agent identity. The agent has unrestricted access to any and all permissions in that API key. And finally, we can't just sort of stick human in the loop everywhere because humans make mistakes too, and many agents also run"
  },
  {
    "startMs": 115700,
    "text": "autonomously."
  },
  {
    "startMs": 119300,
    "text": "So the way we solve this at Keycard is by using an open standard called Token Exchange. This ex- this standard has actually been around since twenty twenty, uh, and it extends OAuth too, and I am going to show you what we're going to be doing today."
  },
  {
    "startMs": 136960,
    "text": "So in order to give identity to an agent and the user behind the agent, we need to know who the user is as the first order of business. So an authorization server verifies identities and issues tokens. Identity providers like Google, Okta, Auth0, et cetera, provide authorization servers. And if we want to get an identity chain in the agent execution path, then we need the user to log in first."
  },
  {
    "startMs": 162420,
    "text": "After logging in, the authorization server prompts the user for their consent to delegate some portion of their access to resources with a subset of their permissions called scopes. Now, this is kind of the first narrowing of access, right? We're only delegating some of the user's total access to the agent."
  },
  {
    "startMs": 181920,
    "text": "And the authorization server issues a token id- that identifies the user and includes their level of access."
  },
  {
    "startMs": 190820,
    "text": "So now if we go into what actually happens in the agent execution path. We have an authenticated user now with a known identity who wants to use an LLM to interact with resources. Now, an agent is a control loop that calls an LLM, and in this workshop, that agent is an Expense Desk agent. That's what we're going to be working with."
  },
  {
    "startMs": 213060,
    "text": "Is going to call in an expense MCP server, which sits between the agent and a ledger API. This provides the agent with a defined set of tools, and it also provides the connection to that downstream resource."
  },
  {
    "startMs": 227820,
    "text": "Now, an MCP client takes the agent's proposed tool calls, and it dispatches them to its MCP server."
  },
  {
    "startMs": 236040,
    "text": "The runtime is a process that runs the agent loop and executes calls. So in our case today, the Expense Desk application is the runtime."
  },
  {
    "startMs": 246840,
    "text": "We're gonna use a LangChain agent that's implemented in this custom Expense Desk app."
  },
  {
    "startMs": 253200,
    "text": "So if we follow a prompt through this execution path, the user submits the prompt to the runtime, which calls the model and sends it to the LLM. The model proposes tool calls, which are then dispatched by the MCP client to its MCP server, which executes the tools and then calls the resource API."
  },
  {
    "startMs": 273080,
    "text": "The API returns a response to the MCP server. The MCP client delivers the response to the runtime, which calls the model, and this loop repeats until the model is satisfied and returns a response to the user."
  },
  {
    "startMs": 290600,
    "text": "Now, if we remember, we have an authenticated user with a token identifying who they are, and this token becomes what we call a subject token. It identifies the subject on whose behalf the agent is going to act. So in order to support token exchange, we need an OAuth client that's capable of executing code. So that might be a gateway between the MCP client and maybe a third-party MCP server or a custom agent app or a CLI wrapper around an off-the-shelf coding agent. In this case today, the"
  },
  {
    "startMs": 320360,
    "text": "Expense Desk agent is a custom agent app that implements token exchange for access to the expense MCP server."
  },
  {
    "startMs": 329060,
    "text": "This MCP server also supports token exchange for the next downstream resource, which is the ledger API. This connects to the database where the expenses are actually going to be stored."
  },
  {
    "startMs": 341320,
    "text": "Now, the prompt and the subject token are sent to the Expense Desk agent app. The agent loop runs, and the model proposes a tool call."
  },
  {
    "startMs": 352160,
    "text": "The Expense Desk agent authenticates with Keycard's security token service. Now, this is a special authorization server, and the Expense Desk agent can authenticate using its client credentials or a workload identity. It also sends the user's subject token that declares the user's identity and their level of authorization. Now, the Expense Desk OAuth client creates a token exchange request asking permission to access the MCP server for that"
  },
  {
    "startMs": 382076,
    "text": "tool call, but only that one specific tool call. So now we basically have three pieces of information."
  },
  {
    "startMs": 389856,
    "text": "We have the identity of the agent that's requesting access. We have the identity of the user on whose behalf the agent is acting, and we have the level of access that the user granted to the agent."
  },
  {
    "startMs": 403716,
    "text": "So now we need to decide if the requested token should be issued or not, and we can do this with policy, which is evaluated against what access is requested from what resource, who the agent is, and who the agent is acting for."
  },
  {
    "startMs": 420456,
    "text": "Now, if the delegation chain and the requested access all fall within the policy, then Keycard issues an access token that can be only, only be used at the MCP server."
  },
  {
    "startMs": 431636,
    "text": "Now, this issued token usually expires within minutes, and you should never store it."
  },
  {
    "startMs": 438236,
    "text": "The token is sent to the Expense Desk MCP client, which makes a tool call to the Expense MCP server using that token as a bearer credential. The MCP server validates that token and then uses it to authorize a tool call."
  },
  {
    "startMs": 453836,
    "text": "The tools need access to the ledger API resource. So at that point, the MCP server then initiates a second hop. This is where it sends its own token exchange request to Keycard. And the request to exchange that token is received from the ins- Expense Desk"
  },
  {
    "startMs": 473656,
    "text": "for a token to access the ledger API, which is the downstream resource. Now, Keycard evaluates the request against policy and then mints the credential for the resource scoped only for the access that's needed for that specific tool call."
  },
  {
    "startMs": 491616,
    "text": "The MCP server receives this token, and then it uses it to call the ledger API."
  },
  {
    "startMs": 498716,
    "text": "The API validates the token and then grants the appropriate requested access. Now, the MCP server never stores the token, and it discards it as soon as the call is done."
  },
  {
    "startMs": 509736,
    "text": "The API response then flows back up, and the agentic loop repeats until the model returns the answer to the user."
  },
  {
    "startMs": 519616,
    "text": "So at a high level, this is what you're going to see us implement today with Keycard. And I'm going to pull up this slide again. So again, I'm going to basically demo, or Mason and I are gonna demo everything step-by-step on screen. Uh, you are free to follow along, or if you would prefer to just watch, you can also do that. Uh, the workshop is going to use a pre-provisioned shared Keycard organization. So if you do participate, you"
  },
  {
    "startMs": 549296,
    "text": "are going to be in the same organization as everybody else in the room, so you'll be able to see what everybody else is doing. Okay, so if you go to the link that is pro-- that was provided, you're going to see this. This is the, uh, workshop running in essentially a sandbox."
  },
  {
    "startMs": 571556,
    "text": "So we're gonna go ahead and start this. This is gonna take a few min-- a minute or two to, uh, get everything all set up. But basically, kind of the premise here is we're going to start with an Expense Desk agent that does not require a login. It doesn't know who is acting at any point, and everybody who uses the agent is using an API key. So I have some folks, uh, here who are going to be helping"
  },
  {
    "startMs": 601916,
    "text": "to populate the demo, essentially. Um, if you want to follow along as well, you would-- are going to eventually need, uh, admin access in Keycard. So as we go through the workshop, when we hit that part, uh, they'll be watching to see people show up, uh, in the user list, and they will be granting you admin access as that happens."
  },
  {
    "startMs": 628796,
    "text": "So when you begin the workshop, this is what you're going to see. So basically, this is the application where we don't know anybody's identity. Everybody is using the same MCP server, but they're all running their own agents. They are all using an API key. They might be using the same one. They might be using different ones. We, you know, don't really have a way of knowing that. But if we look at the Expense Desk, it-- we kind of see your, your classical, uh, expense. You file an expense, and"
  },
  {
    "startMs": 658556,
    "text": "then it either gets approved or rejected, um, and then paid out, which is not actually going to happen, so you're not gonna receive money. Um, but basically everything in this Expense Desk operates through the agent. So the first thing we are going to do is we're going to submit an expense."
  },
  {
    "startMs": 683916,
    "text": "So I'm gonna tell the agent I want to submit a seventy-five dollar expense for snacks for the workshop. It submits that expense, and then its status is pending."
  },
  {
    "startMs": 698996,
    "text": "Now, if we go down to the activity here, we can see that, uh, this expense was created using access granted by an API key, right? So we don't have any user identity. We don't know the agent's identity. The agent doesn't really have an identity yet on its own."
  },
  {
    "startMs": 719776,
    "text": "And what I'm going to do is I'm going to refresh this, and I'm going to approve or deny somebody else's expense."
  },
  {
    "startMs": 731064,
    "text": "So let's see this one. If I hit Approve, it's basically, uh, just going to send that as a prompt to the agent."
  },
  {
    "startMs": 741824,
    "text": "It went ahead and approved that expense, and now we have the created by. It was created by this API key. So I don't know who created that expense, right? Everybody's just using API keys. I approved it, but we also don't have any record of who approved it either, because again, all we have is an API key."
  },
  {
    "startMs": 771824,
    "text": "So the next thing that we wanna do is actually give the agent an identity so that it can act on its own behalf."
  },
  {
    "startMs": 785924,
    "text": "Uh, so the first thing we are going to do now is we are going to join that shared Keycard organization. So if you click the link -- If you're following along, if you click the link in the, uh, instructions, you will be able to-- you will get a prompt to log in with GitHub, and you can sign up with GitHub to join Keycard. When you get in, you should say-- see this Workshop AIE NYC in the upper left corner, indicating that you're in the right, uh,"
  },
  {
    "startMs": 817784,
    "text": "organization. And let me move this around so we can see things better."
  },
  {
    "startMs": 833324,
    "text": "Okay. So if you're following along, when you join, you're likely to see a couple of consent prompts. These are what I kind of talked about earlier in the presentation, where you are delegating some subset of your access to the agent to use on your behalf."
  },
  {
    "startMs": 857384,
    "text": "So this is the Keycard console. The first thing I wanna talk about is applications. Now, applications and resources are registrations, so something can be both an application and a resource. And basically what this means is applications are things that are going to be able to make requests for tokens. They're going to ask for token exchanges in order to then exchange on down the line to access other resources. So"
  },
  {
    "startMs": 888324,
    "text": "our-- the first thing we wanna do is we wanna create an application registration for our agent. So what we are going to do here,"
  },
  {
    "startMs": 899624,
    "text": "uh, is"
  },
  {
    "startMs": 902944,
    "text": "I'm gonna call this agent app. Oops."
  },
  {
    "startMs": 909244,
    "text": "And then in order-- Because we're all in the same, uh, organization here, we want to name things so that we can find our own applications and resources that we created. So I'm going to use the-- my GitHub handle."
  },
  {
    "startMs": 924364,
    "text": "Every app's identifier needs to be unique, so we are going to do something similar here."
  },
  {
    "startMs": 932764,
    "text": "We're going to use implicit consent, and then we need to grab our redirect URL. Now, this is something that we can get out of our, uh, sandbox image. So if we go into..."
  },
  {
    "startMs": 950784,
    "text": "I don't know why that one won't come to the front."
  },
  {
    "startMs": 958684,
    "text": "Go to the terminal, and we are going to run this. This gives you the redirect URL that is specific to your instance of the sandbox. So I'm just going to copy that."
  },
  {
    "startMs": 976064,
    "text": "I'm going to... Oops, don't need to. Add this as the redirect URL."
  },
  {
    "startMs": 984904,
    "text": "So after there's a login event, this is where the, uh, user is going to be redirected back to. And I'm gonna say Create application."
  },
  {
    "startMs": 1003244,
    "text": "So now we have a registration that says our Expense Desk agent can request tokens, right? So but our Expense Desk agent can also receive tokens because it's going to receive the subject token from the user when the user logs in. And in order to do that, we basically need to do another registration, and we need to say that this agent application can also act as a resource, which means it is a target recipient for tokens. So the authentication"
  },
  {
    "startMs": 1033304,
    "text": "token is the token that this agent is going to be receiving. So if we go into Resources, we can add another resource. We are going to call this one... The first one was agent app. This one is agent resource.  I'm going to use my GitHub name."
  },
  {
    "startMs": 1054844,
    "text": "Let's use a similar identifier. This is the agent resource."
  },
  {
    "startMs": 1064984,
    "text": "I'm gonna set the credential provider to zone provider, which essentially means that Keycard is the entity that's going to be handling the token exchange and issuing, uh, tokens for this resource."
  },
  {
    "startMs": 1085284,
    "text": "And then I'm going to come down here, and I'm going to say, \"This resource is being provided by my application, the agent app Kay Mada.\""
  },
  {
    "startMs": 1096196,
    "text": "So basically, this is the relationship that tells Keycard that, uh, this application can exchange sign-in tokens addressed to this resource. If we don't make this association, then the exchange requests are gonna be rejected."
  },
  {
    "startMs": 1113756,
    "text": "So I'm going to save that."
  },
  {
    "startMs": 1118576,
    "text": "And now I want to add the things that my agent application depends on. So if I come back to Applications, and I find my application,"
  },
  {
    "startMs": 1132556,
    "text": "I'm gonna go into Dependencies, and then I'm going to look at the list of dependencies, and I'm gonna add something here. So the first one I'm going to add is the agent resource, and we need this because our agent application is going to request token exchanges to access, uh,"
  },
  {
    "startMs": 1149236,
    "text": "downstream resources. We also want to be able to request token exchanges for the expense MCP server. So this is a resource that's already been pre-configured for you. Everybody is using the same one, so we don't need to create a bunch of them that are all named after ourselves."
  },
  {
    "startMs": 1169256,
    "text": "Additionally, we have an agent, and we want to have the API key for that agent stored in Keycard so that we don't have its secret on disk. So Keycard also provides a vault, and we have that API key configured in the vault already for you. So we can select the LLM API. And finally, we're gonna c- select this user info endpoint as well because this is going to be able to tell us who the logged-in user"
  },
  {
    "startMs": 1199276,
    "text": "is. It's gonna be able to provide us th- with that identity information. So I'm gonna say Connect."
  },
  {
    "startMs": 1211896,
    "text": "Now, the next thing I need to do is I need to give this agent app an, its own identity, and I can't really do that unless it has some way to identify itself to essentially log in. Uh, so I'm going to click on Add Credential. I'm going to click on Client ID & Secret."
  },
  {
    "startMs": 1234716,
    "text": "Now,"
  },
  {
    "startMs": 1242796,
    "text": "we're gonna go into the editor. We are going i- into the agent folder. We're gonna find the end file, and we are going to paste in that client ID"
  },
  {
    "startMs": 1256416,
    "text": "and that client secret."
  },
  {
    "startMs": 1264936,
    "text": "Now, and, um, we're doing this because this is essentially like a sandbox local environment. If you were using this in a production environment, you would probably use work-- something like Workload Identity. Also, Keycard provides a CLI which can populate secrets for you dynamically and not require them to be on disk."
  },
  {
    "startMs": 1285636,
    "text": "And then finally, I'm going to tell it what the identifier for the agent resource is. So that was my uran-agent resource and then my GitHub name. Gonna wait until this says Changes saved."
  },
  {
    "startMs": 1307276,
    "text": "And at this point, we have configured Keycard to be able to handle the token exchanges and exchange requests. Now what we need to do is make sure that our Expense Desk application can actually handle that."
  },
  {
    "startMs": 1321696,
    "text": "So essentially, when we go and look at the code here, if we find the-- in the agent folder, we have this agent auth. If we go down and we find... Okay, this thing is in the way."
  },
  {
    "startMs": 1339416,
    "text": "Try to remove that."
  },
  {
    "startMs": 1348636,
    "text": "So we're going to find the"
  },
  {
    "startMs": 1353055,
    "text": "checkpoint which-- So this, this is the file where we don't have any authenticated MCP access, right?"
  },
  {
    "startMs": 1361576,
    "text": "Now, the way the workshop works is if you-- a- at each lesson, we have a checkpoint that will provide the code essentially for the end of that lesson. So what we can do now is we can go and look for the checkpoint for exercise two, which is the one that we're at. So if we go to Checkpoints, we go to Authenticated, and then we find the same file."
  },
  {
    "startMs": 1394116,
    "text": "Where are we here?"
  },
  {
    "startMs": 1405156,
    "text": "Okay. So we wanna find this MCP toolset. This is passing in, uh, the Keycard zone token auth from the Keycard SDK, and this is what is going to actually make that token request every time the agent actually wants to make a tool call with the MCP server. So for every individual tool call, we're going to get a token that's scoped to that specific tool call."
  },
  {
    "startMs": 1441176,
    "text": "Now, this call is the..."
  },
  {
    "startMs": 1448776,
    "text": "What is this?"
  },
  {
    "startMs": 1460504,
    "text": "Okay. So now if we go to the Keycard file,"
  },
  {
    "startMs": 1469024,
    "text": "we look for this mint function. This is what actually goes ahead and makes that request to mint a token for the exchange. So if we don't have a subject token, that is if we don't have a logged-in user, then the agent is actually going to authenticate as itself using a client credentials grant. So it's going to use that, uh, client ID and client secret to identify itself to"
  },
  {
    "startMs": 1498744,
    "text": "Keycard, and this means it's acting on its own behalf."
  },
  {
    "startMs": 1508444,
    "text": "Okay. So now if we go to the Terminal tab, we basically want to make sure that we activate this updated code."
  },
  {
    "startMs": 1518804,
    "text": "So we're going to run that, so it switches over to checkpoint two,"
  },
  {
    "startMs": 1524384,
    "text": "and then we have to restart the services."
  },
  {
    "startMs": 1530544,
    "text": "Stop. There it goes."
  },
  {
    "startMs": 1534664,
    "text": "Okay. Now if we go and look at the Expense Desk,"
  },
  {
    "startMs": 1542664,
    "text": "we have a screen that will let us continue with an authenticated Expense Desk agent. So if we go ahead and then click continue as the agent,"
  },
  {
    "startMs": 1554804,
    "text": "then we're gonna ask it, uh, which-- I can just copy this. Which account is being used right now, and what is my approval limit? And the approval limit is essentially the amount of money that you as a user are allowed to approve for other people's expenses."
  },
  {
    "startMs": 1575264,
    "text": "So it says I'm signed in as the Expense Desk agent, K Maeda, which was in Keycard. That was the, uh, specific application that we are acting as, that we provided the client ID and secret for."
  },
  {
    "startMs": 1590604,
    "text": "And the approval limit for the agent is about fifty dollars."
  },
  {
    "startMs": 1597124,
    "text": "So next, I am going to tell the agent, uh, file a seventy-five dollar expense"
  },
  {
    "startMs": 1605884,
    "text": "for, let's just say,"
  },
  {
    "startMs": 1613104,
    "text": "for some swag for the workshop on my behalf."
  },
  {
    "startMs": 1623844,
    "text": "So the agent submitted that expense."
  },
  {
    "startMs": 1628424,
    "text": "The memo says it was f-filed on behalf of Kim, but the actual activity details indicate that this was filed by this specific agent."
  },
  {
    "startMs": 1642724,
    "text": "So the on behalf of Kim is just something I told it. It doesn't necessarily, uh, actually have any way to verify that because I, as a user, have not signed in. It created this expense under its own identity."
  },
  {
    "startMs": 1659884,
    "text": "Try to get this."
  },
  {
    "startMs": 1670344,
    "text": "Okay. So now if we go into our applications... Actually, probably could have left that up. Oh, it's right there."
  },
  {
    "startMs": 1681084,
    "text": "We can see the activity in Keycard. This is essentially the audit log that Keycard generates of all the activities that have been happening against this application."
  },
  {
    "startMs": 1691904,
    "text": "And I am going to"
  },
  {
    "startMs": 1698044,
    "text": "look for specifically, because there's, uh, several different resources here."
  },
  {
    "startMs": 1711964,
    "text": "To look for the expense MCP resource, anything that was calling the expense MCP resource. I'm gonna limit this to the last hour."
  },
  {
    "startMs": 1732084,
    "text": "And we can see that there was a credential issued using the client credentials grant, which means that the actor was the application itself, and it was accessing the MCP resource."
  },
  {
    "startMs": 1748604,
    "text": "So now I'm going to go and look for what we have already registered for you, which is the expense MCP actor. So the expense MCP server has multiple registrations as well because it can receive tokens, uh, from the agent application, and then it can exchange that token on for a downstream token to access the ledger API."
  },
  {
    "startMs": 1778964,
    "text": "So if we click on this one,"
  },
  {
    "startMs": 1782584,
    "text": "now we have-- Because everybody is using the same resource for the MCP server, we are going to go ahead and just filter it down"
  },
  {
    "startMs": 1800884,
    "text": "to it calling the ledger API."
  },
  {
    "startMs": 1809704,
    "text": "And I could technically use any of these, but..."
  },
  {
    "startMs": 1819604,
    "text": "Oops, this is looking for people"
  },
  {
    "startMs": 1828848,
    "text": "So if we look at one of the calls that the expense MCP server used, executed in order to exchange the token that it received from the agent application to access the ledger API, we can see that the delegation chain shows that the application that made the request was the expense MCP server, and that it was acting on behalf of this application. And the resource it is accessing"
  },
  {
    "startMs": 1858628,
    "text": "is the ledger API. So this is essentially the, the record of the delegation chain for the agent acting as itself."
  },
  {
    "startMs": 1870948,
    "text": "I'm gonna go ahead and check that."
  },
  {
    "startMs": 1882808,
    "text": "Okay. So now we have an agent that can a- identify itself and act as itself, but we maybe want it to act on our behalf, but without impersonating us. So we don't want it to just say, \"Kim did this.\" We want it to say, \"This agent did this thing on behalf of Kim with Kim's permissions.\" So the next thing we are going to do is we are going to add human user identity into this process."
  },
  {
    "startMs": 1912488,
    "text": "So now we have user delegation. At this point, we need to load the Expense Desk in a new tab because the iframe that's embedded into the sandbox doesn't handle external authentication, um, so we are going to go ahead and open that in an external window."
  },
  {
    "startMs": 1935948,
    "text": "This down here."
  },
  {
    "startMs": 1943808,
    "text": "And now we have this Sign In button up on the right. I'm going to click that. If it's..."
  },
  {
    "startMs": 1953588,
    "text": "It's thinking."
  },
  {
    "startMs": 1956588,
    "text": "Okay. So because I was already logged in, it went ahead and logged me in automatically. Um, but now what I can do is I can say-- We can see that my email is here. If I ask the agent which account am I using now, before it said, \"You're us- you're acting as this agent.\""
  },
  {
    "startMs": 1980388,
    "text": "Now it knows that I have an identity because I authenticated as a human user, right? And my approval limit, which is associated with my identity metadata, is a hundred dollars."
  },
  {
    "startMs": 1999688,
    "text": "So I haven't filed any expenses under my human user yet, so if I change this filter to show all expenses,"
  },
  {
    "startMs": 2010368,
    "text": "you can see that these expenses from earlier still, um, say that they were submitted by the API key. Like, adding all of this doesn't go back and change anything that's happened in the past. And we can see, you know, examples of the Expense Desk agent acting on behalf of other people as well."
  },
  {
    "startMs": 2031768,
    "text": "So we have the ability now to act as ourself as a human user, but we have still retained the ability for the agent to act as itself as well. And the way that kinda plays out in this workshop is by the agent having the capability to approve expenses that are under a certain amount of money."
  },
  {
    "startMs": 2056988,
    "text": "So if I... I'm just gonna do this one."
  },
  {
    "startMs": 2063308,
    "text": "If I submit an expense for, say, thirty dollars for office supplies for the team,"
  },
  {
    "startMs": 2072328,
    "text": "it's going to verify who I am. It's going to file the expense."
  },
  {
    "startMs": 2087507,
    "text": "And then because the agent actually has the ability to approve expenses under forty-nine ninety-nine on its own, it filed the expense, and then the agent approved it as itself. So if I click on the activity to examine what happened here, it says that this expense was created by me under my email,"
  },
  {
    "startMs": 2111948,
    "text": "and then it was approved by the Expense Desk agent."
  },
  {
    "startMs": 2117488,
    "text": "So it shows, you know, th- this is the identifier for the agent that acted in the approval, and it gives a reason. Says, \"Eligible office supplies under fifty dollars with a clear business purpose.\""
  },
  {
    "startMs": 2130488,
    "text": "Uh, so now maybe I want to..."
  },
  {
    "startMs": 2136948,
    "text": "Mm, let's do this seventy-five dollar one as well."
  },
  {
    "startMs": 2148048,
    "text": "So if I submit an expense that is over that approval limit that the agent can act on on its own, then the status for that one is pending. A human needs to review it."
  },
  {
    "startMs": 2159788,
    "text": "If I go and say, \"Hey, uh, I think this should be approved,\" so I'm going to send the message to the agent that's, that says, \"Hey, approve expense one two three,\""
  },
  {
    "startMs": 2177008,
    "text": "it says, \"I can't approve or reject my own expense, so I have to ask another reviewer to do that.\" And it's able to do that because we have identities now. We know who all of the users are, who are logged in, and we also know the identity of the agent itself."
  },
  {
    "startMs": 2193560,
    "text": "Uh, so before we didn't demonstrate this, but before I could have approved my own expenses or anybody else's because it didn't know who anybody was."
  },
  {
    "startMs": 2204000,
    "text": "So if I go here, let's see."
  },
  {
    "startMs": 2213760,
    "text": "Let's find somebody else's pending expense."
  },
  {
    "startMs": 2222220,
    "text": "I'm going to say, uh,"
  },
  {
    "startMs": 2226460,
    "text": "I wanna reject this one."
  },
  {
    "startMs": 2244560,
    "text": "The agent goes through that song and dance, and now if we look at the recorded activity for this expense, it says that it was created by the expense desk agent, uh, because it was from kind of the previous cycle."
  },
  {
    "startMs": 2258540,
    "text": "And it was rejected by my identity."
  },
  {
    "startMs": 2264800,
    "text": "So now we have a, kind of a clear record of who filed what, who approved or rejected what."
  },
  {
    "startMs": 2276480,
    "text": "So if we go into the Keycard console at this point."
  },
  {
    "startMs": 2281280,
    "text": "Go to applications. We are going to find the Kmaida application."
  },
  {
    "startMs": 2289700,
    "text": "I'm gonna clear any previous filters. I believe they're already gone. I'm gonna say actor, and I'm gonna look for my email."
  },
  {
    "startMs": 2301400,
    "text": "Change this to last hour."
  },
  {
    "startMs": 2306560,
    "text": "And if I click on one of these issued credentials,"
  },
  {
    "startMs": 2311620,
    "text": "I can see that the application that, uh, made the token exchange request was my application agent app Kmaida, and it was acting on behalf of me, the human user, Kmaida Keycard or Kmaida Keycard, and it was accessing the expense MCP resource. So that was the first hop, right? So now if we go over to"
  },
  {
    "startMs": 2337860,
    "text": "the applications again, and we wanna find the expense MCP actor."
  },
  {
    "startMs": 2352640,
    "text": "Actually, I don't think we need to filter anything. It's all here. And we go to a credential that was issued for access to the ledger API. We can now see that on the second hop, the expense MCP server made the request on behalf of me, and it was accessing the ledger API."
  },
  {
    "startMs": 2373760,
    "text": "So on the first hop, what happened is the agent took my sign-in token that it acquired when I logged in as my human user, and it exchanged the-- it made a token exchange request to Keycard offering that sign-in token in exchange for a token for the next thing in, in the, uh, chain, right? Which is the MCP server. So it exchanged the subject token for an MCP server token. It then took that MCP server token and called the"
  },
  {
    "startMs": 2403780,
    "text": "MCP server to make a tool call with the appropriate tool scoped scopes. And then the MCP server took that token that it had received from the agent, and it made a token exchange request to Keycard asking to give that token up for a, a token to access the ledger API, which was the next resource in the stream."
  },
  {
    "startMs": 2430640,
    "text": "And you can trace these hops..."
  },
  {
    "startMs": 2436000,
    "text": "You can trace these things across hops because they have a session associated with them. So we're not going to spend a whole lot of time, like, digging into the, the session numbers and copying them, but you can s- follow any specific sequence of events, uh, by the session number will be consistent on the first hop and also the second hop."
  },
  {
    "startMs": 2468220,
    "text": "I think we can close all that. And I will now give it up to Mason."
  },
  {
    "startMs": 2474980,
    "text": "Awesome. Thank you so much, Kim. Uh, I have my little plushy tardigrade. Ziggy goes with me everywhere. We have plenty of them, so make sure you grab one before you leave. Take one home for your kids or for yourself. No shame. Um, whatever you'd like to do. So we will just, uh, walk through, uh, how to build all of this security and stuff into our agent and stuff. But now how do we deploy these agents to ensure that they execute to their completion, that they don't fail, or when they do fail, we know how to handle that, and we know how to, uh, reliably bring them back up so we don't lose all of our process and waste a whole bunch of time and"
  },
  {
    "startMs": 2505000,
    "text": "compute. And that's, uh, where Temporal comes into play.  So Temporal is a durable execution framework. So essentially what it does is we take your code. We execute it. We maintain the state of the application. If the application were to fail, run into an error, things of that nature, it will reconstruct the state or rebuild the state to the point of failure and continue executing onwards. Certain things such as, uh, function calls and things, if, say, a distributed system were to go down, there are built-in retries and all sorts of failure handling modes."
  },
  {
    "startMs": 2535020,
    "text": "So one of the things that we've seen a lot of, um, with the customers that I work with and with our users is that, uh, we need a way of making your agents durable. And in reality, the agent loop is-- or the agent itself is just a loop. It's a, it's a loop of ever-calling tool calls to external services. Agents are just distributed systems at the end of the day, with a little bit of added spice on top with some thinking. But the problems that we've solved for distributed systems apply to agent loops, and this is where Temporal comes in and works out really well to keep your application"
  },
  {
    "startMs": 2564952,
    "text": "running and going so you don't lose data, so you don't have to re, uh, call LLMs, uh, wasting tokens and all that nature. So you, um, so your users don't experience any sort of outage, all sorts of really awesome things with that. So what we're gonna do a little bit is just kinda talk about a little bit of how you would structure a Temporal application code. I'll leave this here for you to read later. I'm not gonna dive super deep into the code itself. But essentially, Temporal is a, a system where you can-- you build a workflow, and you define it by things what are called activities. Think parent function and child function calls. Easiest"
  },
  {
    "startMs": 2594972,
    "text": "way to think about it. We implement our loop, uh, as a-- this parent function call, and this becomes a workflow. And every single, uh, instance of or execution of this, uh, workflow, so like every single time we would have Keycard go through and, uh, authenticate for user, that would be a workflow. And every single tool call is essentially an activity. These are things that can fail, that we expect to fail, and when they do fail, we either retry them or we rebuild the state and see what happens with it. So we have our workflow calls here. I'm also-- I do the other direction for Mac, Mac mouse. So if I go the wrong direction, it's 'cause I'm"
  },
  {
    "startMs": 2625672,
    "text": "not, not using my laptop. Um, I remap everything on my computer. Uh, and then down here, you actually see, like, this async def assess. This is actually, uh, what an activity would look like. So y-- it looks like normal Python code. It looks like the typical code you would write. Temporal is a code-first system, so you can build all of your things and hook it into anything that you would want. Um, so we have our settlement code workflow, which I'm not gonna read through too much. I'll leave that to you, for you. But again, it's a Temporal workflow that, uh, essentially allows us to set how often we wanna retry, and then we execute the activities."
  },
  {
    "startMs": 2655652,
    "text": "We have our, um, our debit ledger right here and our mark settled. So we're able to, you know, add something to the ledger and then mark it as settled. And we have this timer here in the middle for us to kind of simulate some, like, uh, human in the loop time that would wait-- take for someone to do this. Uh, and then one of our activities, uh, for this, as you can see here, is, uh, this section right here, which is just the typical code. But I wanna draw your li-- your, uh, eye to the, to the first decorator on top, which is grant, uh, resource. So Temporal exposes a lot of native, uh,"
  },
  {
    "startMs": 2685572,
    "text": "primitives within its code base, uh, one of which is what's called an interceptor. So it allows you at certain points within the code to allow for essentially-- think of them as hooks. So this is actually something that the Keycard team, uh, implemented, which is where whenever this, um, this activity is being executed, it actually goes and gets permission and gets an identity from Keycard, and then comes back and essentially now we're able to authenticate this through, uh, our Temporal workflow and our Tem-- our activities through this, and we can maintain this identity separation that Kim was talking about, where you can have each"
  },
  {
    "startMs": 2715472,
    "text": "individual worker have its own one. You could have a fleet of them have their own, but you'd be able to identify and limit scope, access all of those wonderful things, uh, through there. Um, so the next thing we're gonna do is we're basically going to find, uh, our work-- I think we already did the workers identity in Keycard. Yeah. Okay, so we're gonna start our worker. So we actually already set up our Temporal application and set up all of these, uh, environment variable configurations. So let's go ahead and start, uh, our worker. So Temporal is a worker-server model, a, a version of execution. You-- there is a Temporal service that"
  },
  {
    "startMs": 2745332,
    "text": "maintains the execution, the application state, all of that. And then there's a fleet of workers. You can think Kubernetes pods, processes within a pod that execute your work as it's being, uh, as it gets-- as the request comes in. So from here, we'll go to start worker, and we'll just start our first worker right here. And so we have worker PID five seven two running. So now we have a, uh, automatic, uh, a Temporal worker that's running the agent that was just done in the previous thing-- in the previous exercise, which will automatically, uh, accept our, uh, expenses. So we have"
  },
  {
    "startMs": 2775352,
    "text": "our worker running right here, and we see our PID is five seven two. And then we can start a settlement. So what we're gonna do is we're gonna click on this client tab here, and we're going to invoke a settlement, uh, that's gonna happen. And we'll see our workflow ID settlement came up. So if we go back to our worker tab here, we'll actually see the output of our app-- Temporal application as it's logging out what it is doing, and it's currently doing, uh, as you can see here, we debited a token, um, and minted it for this particular call, and then we're running it. If we go to our Temporal web UI, and we refresh this"
  },
  {
    "startMs": 2805352,
    "text": "here, we can actually see our application running. I have to do this a little quickly, and then I can continue talking. Oh, no, it's not contr-- Is it? There it is. Okay. Command + Control + C on Macs is not fun for me. So we had our application running. We set-- We had our debit ledger. I killed the application cold. So as you can see here, the running state of this application is gone. We no longer have it. In a traditional system, this would be potentially problematic. We would have to, um, you know, either have saved that state to a database and then re, uh, rehydrate"
  },
  {
    "startMs": 2835272,
    "text": "it when it comes back online. But, uh, with Temporal, that's not really a problem because to-- the state of our application is stored right here inside of it. So the very first, uh, the debit ledger, uh, function stored the output right here. So we have a retry policy, uh, activity task completed. We debited forty-two hundred. So we have that done. There's nothing running right now, so it's gonna start complaining and telling me, \"Hey, you have nothing running. Our workflow task has timed out.\" We have a dead application. Uh, and then we go back to our worker mid-settlement. Whenever we're ready, we can just restart the worker."
  },
  {
    "startMs": 2867732,
    "text": "And the application picks up as if the con-- if the failure never happened. It rebuilt the entire state of our agent from all of the previous responses. No LLM calls were redone. No previous activities or functions were re-executed. It just picked up the state, replayed the actions, and then pushed it through, and we got the execution to work."
  },
  {
    "startMs": 2887612,
    "text": "So why does durability matter for agents? Well, your agents are loops. Um, there's a lot of things inside of your agents right now that cost a lot of money. Making LLM calls on, on a massive scale is not cheap. There's are tokens you have to deal with. There's a lot of failure states. There's a lot of advanced states. Say you get through a really long loop and, you know, you're twelve steps in and you lose the entire state of that application. Now you either have-- you've either had to built-- basically built a version of Temporal yourself where you are maintaining the state of your application in a database somewhere, and then you have to build all the reconstruction logic and all the"
  },
  {
    "startMs": 2917372,
    "text": "retry logic. Or, uh, you can use something like a durable execution system like Temporal, and it will do all of this for you. So this allows for you to have your agents go offline, fail Your LLM provider can go down. You can lose an entire region in AWS. It doesn't matter when they, when those-- when that compute comes back, when those resources come back online, the agents pick up right where they left off at the exact last spot and continue execution. And if it happens quickly enough, if it's a quick enough failure, your user will never even be aware that they, that they lost service of this. Um, if you had been on the receiving"
  },
  {
    "startMs": 2947404,
    "text": "end of this worker call that I had just done where we waited a minute and you were-- knew you were gonna wait a minute for the approval, and that failure happened, and it hap-- and I restarted the worker within, you know, five seconds of the failure, as a user on the client side, you probably never would even notice that there was this kind of failure. And that's what Temporal does, is it brings it all the way out, uh, for you and just keeps the execution going forward regardless. Now, the interesting thing is, um, the secrets for all of this were actually not inside of our Temporal event history. So if we go back to our client over here,"
  },
  {
    "startMs": 2977524,
    "text": "and we run this command-- Oop, I hit a button. Oh, I have to actually put in the correct workflow ID. If I don't, it doesn't work. If I go back here and collect this workflow ID,"
  },
  {
    "startMs": 2989584,
    "text": "uh, it was on the Client tab. Yeah. And we click the up arrow."
  },
  {
    "startMs": 2996564,
    "text": "Now we're gonna hope I know how to paste on Mac. Aha, we got it. Good. So we can actually see-- We actually have a little, a little Python script here that says that no JWT token, uh, material, uh, was basically sent. We didn't-- Like, our worker, through the entire event history here, um, you can see the re-- the, the starting process and the results. So if we click in, I can see the debit ledger here with the result down here. So we have our, uh, each individual function is stored there. So our second function here, Mark Settled, was also,"
  },
  {
    "startMs": 3027064,
    "text": "uh, settled as well. So this was the result of it. But nowhere in our workflow, and if I go to the actual event history, which is-- this is actually what the event history looks like under the hood. You will not find it anywhere, uh, because we didn't store it. Um, so Temporal gives you basically two different ways, uh, to keep the sensitive data out. So we have the interceptor, which is what Keycard provided for us. So they built an interceptor that basically injects it at, uh, ba-- uh, keeps the data and the credentials out of it entirely and then injects it back in when it's time to execute. Think of it like almost like just-in-time credentials, uh, and does"
  },
  {
    "startMs": 3056844,
    "text": "that. Or the other thing for Temporal that's really awesome is there's what's called a payload, uh, encoder. So what you can do, as you saw in our Temporal web UI, your data was, um... Oop, there we go. Wrong button. Your data's just sitting here in nice, wonderful plain text. Um, uh, you have the ability to encrypt this yourself. So you can basically spin up, there's like a, there's an entire, uh, pattern and library in Temporal where you spin up a sidecar, you create an encryption algorithm, you set what the encryption policy is, and all data that comes to Temporal, whether it's"
  },
  {
    "startMs": 3086324,
    "text": "self-hosted or in the cloud, is, uh, essentially encrypted. And we, uh, can't read it. We don't control your keys. We honestly don't quite care about what you're doing inside of it. That's not the part that we care about. We care about putting all these, uh, fancy things in order. So that's a quick overview of, like, Temporal and building out it, uh, building out agents with it. There's a lot of integrations with Temporal you can try out. There's OpenAI, Pydantic AI. Um, I know there's LangChain and a whole bunch of other ones too as well. Um, and I think we even now have, like, an agen-agentic harness now that we've been"
  },
  {
    "startMs": 3116264,
    "text": "building out that's going out too. So you can come by and see us at our booth if you wanna learn more about that. Um, and yeah. So the expense desk, you know, basically we-- as-- I'll bring Kim back up to kind of wrap it up, and we can take some questions if you want. Build-building out the agent identity, building out all of the, uh, agent worker and all of that stuff, and then Temporal becomes the, uh, execution layer to make sure that your agent runs properly, uh, and doesn't fail in the case of, you know, any sort of cataclysmic failure, uh, API outage, AWS region just decides it wants to disappear for a day,"
  },
  {
    "startMs": 3146464,
    "text": "which does happen. So yeah, that's all I have."
  }
]
