# Beyond the Model Card: The Jagged Frontier of AI in Finance

Rayan Krishnan · AI Engineer NYC 2026 · October 13

[Watch recording](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18418s)

[05:06:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18419s) Hi everyone, I'm Rayan, I'm the CEO and one of the co-founders of Vals AI.

[05:07:03](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18423s) I'm going to talk a little bit about going from beyond the model card, how evaluations

[05:07:07](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18427s) can be used to measure this jagged frontier, a special focus on finance for this audience

[05:07:13](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18433s) as well. Yeah, so first off, talking about model cards themselves, we've built some of the most

[05:07:18](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18438s) important benchmarks over the last couple of years that the major foundation model labs have included

[05:07:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18442s) in their model releases. So people, I think, are familiar with artifacts like this. Last week,

[05:07:27](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18447s) we have Sundar talking about our VALs index on stage. And then even just yesterday, Muse models

[05:07:32](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18452s) was citing our finance agent benchmark. You know, it keeps our lives interesting to have to keep

[05:07:37](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18457s) re-updating these slides as new models come out. I had to double check level numbers. You can get

[05:07:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18461s) sense of how capable a model is for valuable work, but that's actually driving value for

[05:07:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18467s) a lot of companies.

[05:07:48](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18468s) I think two interesting stats that jump out at me, the groups that are able to actually

[05:07:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18472s) track AI cost and spend are much, much more likely to have inferior performance.

[05:07:56](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18476s) We also have refining the X-axis, again, it's not immediately obvious which is number one

[05:08:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18480s) or what will be consistently two or three.

[05:08:02](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18482s) In the X-axis, you've got cost.

[05:08:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18484s) In particular, it's cost per task, not just tokened Excel models.

[05:08:11](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18491s) that are core to results may be unintuitive.

[05:08:13](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18493s) When we actually build benchmarks,

[05:08:15](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18495s) one of the most important things that we need to look at

[05:08:17](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18497s) is the metric we use to report accuracy.

[05:08:19](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18499s) And so you have a different takeaway

[05:08:21](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18501s) if you pick a different metric.

[05:08:23](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18503s) A lot of benchmarks deal with partial credit,

[05:08:25](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18505s) so what this is answering is,

[05:08:27](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18507s) in expectation, if I were to give a question

[05:08:29](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18509s) or a task to a model,

[05:08:30](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18510s) what percentage of the points would I expect to see satisfied

[05:08:33](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18513s) by the work output of that model,

[05:08:37](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18517s) especially compared to humans?

[05:08:38](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18518s) And then the fully correct answers metric

[05:08:40](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18520s) is more looking at, of the percentage of questions

[05:08:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18523s) that I give to models, what percentage of them

[05:08:45](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18525s) will come back perfect?

[05:08:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18526s) And that's a very different metric.

[05:08:48](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18528s) It may mean that a model is very good at producing

[05:08:51](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18531s) an incremental response that you can work with

[05:08:53](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18533s) to build a better response,

[05:08:55](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18535s) but if you were to pass this directly to a client,

[05:08:58](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18538s) it would be a coin toss of whether it was

[05:09:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18540s) meeting all the criteria that you have.

[05:09:02](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18542s) And so picking the right metric is very important

[05:09:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18544s) to accurately representing the performance of the model

[05:09:06](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18546s) and actually forming a good intuition of what it can do.

[05:09:10](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18550s) There's another three components that we think a lot about.

[05:09:13](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18553s) So validity, are you actually evaluating what's going on in the real world?

[05:09:16](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18556s) That means are the questions or tasks reflective of actual work going on or are they contrived?

[05:09:21](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18561s) And then are the rubrics actually describing what we're looking for or the standard of

[05:09:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18564s) excellence we want to see?

[05:09:26](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18566s) Reliability, this is usually a function of the number of tasks or the sample count.

[05:09:30](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18570s) A lot of companies, they have maybe five or six samples and they're kind of vibe checking

[05:09:34](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18574s) new models or products as they get access.

[05:09:36](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18576s) But it's actually really important to have a rigorous approach and have enough tasks and samples to make sure your repeated runs actually show consistent results.

[05:09:44](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18584s) And then last is discriminability.

[05:09:45](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18585s) So we don't think it's that valuable to have benchmarks where all models score 100% or all models score 0%.

[05:09:51](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18591s) Ideally, we can actually discriminate between models and see what the frontier looks like.

[05:09:56](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18596s) In particular, we also want to make sure we're building benchmarks forward-looking.

[05:09:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18599s) So what are the new set of tasks that are even more ambitious than what models can do today

[05:10:03](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18603s) to make sure that there's room at the frontier for us to continue to see model progress as they get better?

[05:10:11](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18611s) Okay, and then some of the high-level problems that we've seen and I think others can and do run into.

[05:10:18](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18618s) The first is that agents solve the test, not the problem.

[05:10:20](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18620s) So this is talked a lot about as reward hacking.

[05:10:23](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18623s) And we've seen a lot of instances where we'll have a sandbox and models will be run with our model proxy.

[05:10:28](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18628s) and sometimes they may try and actually call other models

[05:10:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18631s) within that proxy to escape the sandbox

[05:10:34](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18634s) or make use of the internet.

[05:10:36](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18636s) So it's really important to build the containers

[05:10:38](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18638s) in a way that you're actually testing for what matters.

[05:10:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18641s) Can they do the task at hand

[05:10:42](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18642s) or are they actually leveraging other tools

[05:10:44](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18644s) or maybe even other models to get there?

[05:10:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18646s) We've also seen this issue in finance in particular

[05:10:49](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18649s) where a lot of the times evals are looking at a set of rubrics

[05:10:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18652s) or items that we want to see included in the response

[05:10:55](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18655s) And so models can hedge and give three possible answers, which has a higher chance of satisfying

[05:11:01](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18661s) all the criteria, but in practice that's way too much.

[05:11:05](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18665s) A person does not want to consider three different answers, they just want to get the first answer

[05:11:08](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18668s) and have it be right.

[05:11:10](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18670s) And so the goal here is to try and build evals that are reflective of the actual work people

[05:11:14](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18674s) are doing and the work output they want to see, not to build these contrived evals that

[05:11:19](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18679s) the models can actually gain.

[05:11:21](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18681s) The second is, I think a lot of people are starting to see

[05:11:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18684s) that the cost of software is coming down,

[05:11:26](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18686s) and it's actually very, very easy to produce code.

[05:11:29](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18689s) But unfortunately, we still see in practice

[05:11:32](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18692s) that running evals is incredibly challenging,

[05:11:34](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18694s) especially from an infrastructure side of things.

[05:11:36](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18696s) And in particular, as we get to longer-running agents,

[05:11:39](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18699s) it means that it's really important to have infrastructure

[05:11:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18701s) that can stay consistent over a long period of time,

[05:11:44](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18704s) and then, if interrupted, can be resumed and respun back up.

[05:11:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18707s) We have, I think, just recently crossed six trillion tokens just in evaluation.

[05:11:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18712s) So we've run a considerable set of evals and we run everything in as high-degree parallel as possible.

[05:11:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18719s) So I think this has really been load tested, but I think other people are going to run into this even on a much smaller scale.

[05:12:05](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18725s) And the last is that I think a lot of the existing benchmarks, they deal with a single prompt, single response,

[05:12:10](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18730s) but in practice everything is multi-turn.

[05:12:12](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18732s) You have initial spec which you think covers everything, but then you get questions back

[05:12:17](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18737s) or you have additional feedback once something is built, and then it takes more of this iteration

[05:12:20](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18740s) to actually drive to the end output.

[05:12:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18742s) And so I think benchmarks and evals need to actually cover this multi-turn component.

[05:12:27](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18747s) That's not going to go away.

[05:12:29](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18749s) So when we look at a case study in particular, when we look at our team, we did a little

[05:12:33](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18753s) token maxing experiment where we got free access to some tools and just gave it to all

[05:12:37](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18757s) the engineers, and they really made use of these unlimited tokens.

[05:12:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18761s) and if we actually look back for that month,

[05:12:44](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18764s) we spent $1.5 million worth of tokens.

[05:12:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18767s) This was like a team of I think 10 or 12.

[05:12:49](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18769s) So it was roughly 10 times the amount

[05:12:50](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18770s) we were spending on salary for tokens in that month.

[05:12:53](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18773s) And so that was pretty insane,

[05:12:54](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18774s) but we saw great output.

[05:12:56](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18776s) And so our goal was to try and maintain

[05:12:58](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18778s) the same level of output across all these engineers,

[05:13:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18780s) but massively cut down on cost.

[05:13:03](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18783s) And really our solution to do that

[05:13:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18784s) was of course to build an eval.

[05:13:06](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18786s) We built a bunch of tools internally

[05:13:08](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18788s) and then have now exposed this

[05:13:09](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18789s) as a product called Vals Smith.

[05:13:11](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18791s) This is what we are doing in coding and I think it is a sign of what is going to come

[05:13:15](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18795s) in other parts of knowledge work.

[05:13:17](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18797s) So we were able to look at the GitHub repo as a centralized way of seeing what work has

[05:13:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18802s) already been done

[05:13:23](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18803s) and what work is continuing to happen.

[05:13:25](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18805s) In particular, GitHub repos are comprised of PRs, which have a GitHub issue and a resolution.

[05:13:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18811s) So what we can do is unwind the clock and give the same tasks to agents to see if they could have done the work of humans.

[05:13:38](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18818s) In doing that, we found a much clearer understanding of which models were overkill, way too expensive and did not need to be used,

[05:13:45](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18825s) and then which ones were actually capable of getting the task done.

[05:13:48](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18828s) And I think, again, this is another place where the intuitions

[05:13:50](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18830s) that a lot of people have may not be what's actually borne out in practice.

[05:13:54](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18834s) So I think you may see we're using a lot of closed-source models.

[05:13:56](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18836s) We should try and cut costs and use our own self-hosted open-source model.

[05:14:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18840s) You can see the comparison here we have between GPT-5.6 Sol and Kimi K3,

[05:14:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18844s) and we see that 5.6 Sol is better.

[05:14:07](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18847s) It's going 2% better on this internal eval.

[05:14:09](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18849s) It's also considerably cheaper than hosting it

[05:14:12](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18852s) through a very inference-optimized stack.

[05:14:15](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18855s) So it's $2.50 instead of $11, and it's faster half the time.

[05:14:19](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18859s) And so there's no real reason why this kind of intuitive sense

[05:14:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18862s) of using open source would be a legitimate way

[05:14:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18864s) to drive down costs.

[05:14:25](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18865s) And so this has resulted in two main differences.

[05:14:28](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18868s) One, we can actually figure out which tools we're

[05:14:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18871s) going to procure, what the usage limits of those things

[05:14:33](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18873s) should be across the organization.

[05:14:35](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18875s) But then also, we can give recommendations for engineers

[05:14:38](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18878s) say, we think that for this GitHub issue,

[05:14:40](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18880s) you should actually use this particular coding tool with this model.

[05:14:44](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18884s) And this is able to drive the cost down considerably.

[05:14:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18887s) I think we're spending 90% less, but still able to get the same output.

[05:14:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18892s) I think the question I would have for you all,

[05:14:54](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18894s) we're continuing to expand this approach

[05:14:56](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18896s) and take this model of coding to other domains,

[05:14:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18899s) knowledge work, legal, finance, things like that.

[05:15:01](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18901s) But the key question is, where are you tracking the history of work

[05:15:05](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18905s) that has already been done by that firm?

[05:15:07](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18907s) Some of these law firms have been around for like 100 years.

[05:15:09](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18909s) And so where is that, I think, quality of the organization stored?

[05:15:13](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18913s) Where are you tracking the standard of excellence?

[05:15:16](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18916s) And so today may exist in places like email,

[05:15:18](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18918s) from the history of responses that people have had with their clients,

[05:15:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18922s) may exist in SharePoints, may exist in Databricks dumps,

[05:15:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18924s) but I think this is the key treasure trove that a lot of companies have

[05:15:28](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18928s) and don't realize will be the major unlock for evals

[05:15:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18931s) and then the unlock for AI adoption more systematically.

[05:15:40](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18940s) Once these evals are constructed, they can enable adoption to happen, but we have also

[05:15:45](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18945s) seen that there are more advantageous ways that they can be used.

[05:15:48](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18948s) First is right sizing.

[05:15:51](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18951s) Rather than just token maximizing, how do you figure out what the right usage limit should

[05:15:55](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18955s) be across the organization?

[05:15:57](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18957s) Negotiation, once you realize you have actual leverage, you can take that to the conversation

[05:16:01](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18961s) you have in bulk pricing.

[05:16:03](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18963s) Then routing, this is becoming a core primitive in a lot of AI products, but can also be done

[05:16:09](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18969s) done by users directly, and so they can build intelligent routers that actually figure out

[05:16:14](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18974s) the right model to use.

[05:16:16](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18976s) And then fine tuning or post training.

[05:16:18](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18978s) This can be the most reliable way for you to actually mint the tasks and then have an

[05:16:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18982s) external eval set to make sure that this approach is working.

[05:16:27](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18987s) Last thought I'd leave you with is I think for a lot of companies, they have a competitive

[05:16:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18991s) advantage that has existed in the heads of employees through things like institutional

[05:16:35](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18995s) knowledge.

[05:16:36](https://www.youtube.com/watch?v=akEqiXFo3GM&t=18996s) and what evals allow you to do is actually codify the competitive advantage

[05:16:40](https://www.youtube.com/watch?v=akEqiXFo3GM&t=19000s) and put that down in words or in scripts, in language

[05:16:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=19003s) and that will actually be the biggest unlock

[05:16:45](https://www.youtube.com/watch?v=akEqiXFo3GM&t=19005s) for making sure that a company succeeds the agentic era.

[05:16:49](https://www.youtube.com/watch?v=akEqiXFo3GM&t=19009s) Okay, that's all. Thanks, everyone.