[
  {
    "startMs": 1309,
    "text": "[music]"
  },
  {
    "startMs": 12600,
    "text": ">> All right, hello everyone."
  },
  {
    "startMs": 14600,
    "text": "My name is Will and uh I'm here to talk"
  },
  {
    "startMs": 17760,
    "text": "to you about automated code review. Uh"
  },
  {
    "startMs": 21360,
    "text": "my teammate Amir and I work at Uber and"
  },
  {
    "startMs": 24200,
    "text": "we're going to be walking through U"
  },
  {
    "startMs": 25840,
    "text": "Review, a system that Uber has built uh"
  },
  {
    "startMs": 28160,
    "text": "to help increase the velocity of our"
  },
  {
    "startMs": 30280,
    "text": "software engineering teams."
  },
  {
    "startMs": 33160,
    "text": "Um for a little bit of context about"
  },
  {
    "startMs": 35320,
    "text": "what software engineering org at Uber"
  },
  {
    "startMs": 37760,
    "text": "looks like, we have thousands of"
  },
  {
    "startMs": 39600,
    "text": "software engineers who work across"
  },
  {
    "startMs": 42000,
    "text": "hundreds of teams uh located across 12"
  },
  {
    "startMs": 45360,
    "text": "different sites and uh they work in"
  },
  {
    "startMs": 47720,
    "text": "primarily one of six language-specific"
  },
  {
    "startMs": 50160,
    "text": "monorepos."
  },
  {
    "startMs": 51400,
    "text": "As many of you have probably noticed"
  },
  {
    "startMs": 52960,
    "text": "over the past 24 months, the volume of"
  },
  {
    "startMs": 55800,
    "text": "PRs, the size of PRs has been growing."
  },
  {
    "startMs": 58560,
    "text": "One of the ways that that's been exposed"
  },
  {
    "startMs": 60280,
    "text": "to us has been through uh the metric"
  },
  {
    "startMs": 62960,
    "text": "that we track of the first time to"
  },
  {
    "startMs": 64440,
    "text": "review. Back in 2024, we were seeing"
  },
  {
    "startMs": 67480,
    "text": "that engineers would get their first"
  },
  {
    "startMs": 69040,
    "text": "review within 3 hours."
  },
  {
    "startMs": 71080,
    "text": "Now in 2026, that has grown to 9 hours"
  },
  {
    "startMs": 74480,
    "text": "uh in addition to all of the volume"
  },
  {
    "startMs": 75880,
    "text": "changes. So, in short, code review is"
  },
  {
    "startMs": 79240,
    "text": "now the bottleneck that we are running"
  },
  {
    "startMs": 80880,
    "text": "into."
  },
  {
    "startMs": 81960,
    "text": "Um specifically around automated code"
  },
  {
    "startMs": 84440,
    "text": "review, uh there are there are various"
  },
  {
    "startMs": 86840,
    "text": "options available in the industry, uh"
  },
  {
    "startMs": 88840,
    "text": "but Uber spent the time to invest in"
  },
  {
    "startMs": 91640,
    "text": "building an in-house solution due to"
  },
  {
    "startMs": 93240,
    "text": "some of the constraints that we have."
  },
  {
    "startMs": 94880,
    "text": "One of those is uh we currently use"
  },
  {
    "startMs": 97680,
    "text": "Fabricator and have for a long time and"
  },
  {
    "startMs": 99160,
    "text": "are in the process of migrating to"
  },
  {
    "startMs": 100760,
    "text": "GitHub. Uh most of the solutions do not"
  },
  {
    "startMs": 103360,
    "text": "provide support for Fabricator. Um in"
  },
  {
    "startMs": 106160,
    "text": "addition, if you were at the previous"
  },
  {
    "startMs": 107480,
    "text": "talk, you saw Uday and Adam talking"
  },
  {
    "startMs": 109960,
    "text": "about the agentic SDLC. A big part of"
  },
  {
    "startMs": 112920,
    "text": "what we want to do is bring a consistent"
  },
  {
    "startMs": 114840,
    "text": "code review experience to the inner loop"
  },
  {
    "startMs": 117800,
    "text": "so that our agents are getting the same"
  },
  {
    "startMs": 119320,
    "text": "code review, the same rules, everything"
  },
  {
    "startMs": 121640,
    "text": "applied as our humans do."
  },
  {
    "startMs": 124479,
    "text": "With hundreds of teams across the"
  },
  {
    "startMs": 126240,
    "text": "company, we can't have centralized"
  },
  {
    "startMs": 128600,
    "text": "management of our code reviews, our"
  },
  {
    "startMs": 130679,
    "text": "customizations, and our rules, and even"
  },
  {
    "startMs": 133120,
    "text": "the knowledge that goes into those code"
  },
  {
    "startMs": 134720,
    "text": "reviews. We need to distribute that. So,"
  },
  {
    "startMs": 136920,
    "text": "we have a need for"
  },
  {
    "startMs": 138520,
    "text": "plugging into existing team ownership"
  },
  {
    "startMs": 141000,
    "text": "system rather than trying to replicate"
  },
  {
    "startMs": 143720,
    "text": "that externally."
  },
  {
    "startMs": 145600,
    "text": "Uh finally, with the volume of code"
  },
  {
    "startMs": 147959,
    "text": "reviews that we perform, we need the"
  },
  {
    "startMs": 149640,
    "text": "ability to take factors like the risk"
  },
  {
    "startMs": 152440,
    "text": "profile and the complexity of a code"
  },
  {
    "startMs": 155920,
    "text": "change and factor that in when deciding"
  },
  {
    "startMs": 158440,
    "text": "how we're going to run a code review."
  },
  {
    "startMs": 160440,
    "text": "Not all code gets the exact same review."
  },
  {
    "startMs": 163120,
    "text": "And then finally, consistency. We need"
  },
  {
    "startMs": 165040,
    "text": "to make sure that we have security and"
  },
  {
    "startMs": 166680,
    "text": "compliance reviews run across"
  },
  {
    "startMs": 168360,
    "text": "everything. We can't rely on teams"
  },
  {
    "startMs": 171600,
    "text": "hoping to run the skill the code review"
  },
  {
    "startMs": 173840,
    "text": "skill that happens. We need reliability"
  },
  {
    "startMs": 175480,
    "text": "there."
  },
  {
    "startMs": 177000,
    "text": "With all that said, I wanted to give you"
  },
  {
    "startMs": 178920,
    "text": "an overview of the architecture of what"
  },
  {
    "startMs": 181400,
    "text": "you review looks like. We'll talk about"
  },
  {
    "startMs": 183040,
    "text": "a couple of the big pieces, and then"
  },
  {
    "startMs": 184720,
    "text": "we're going to dive into a few focus"
  },
  {
    "startMs": 186200,
    "text": "areas."
  },
  {
    "startMs": 187920,
    "text": "At the top, you'll notice that we have"
  },
  {
    "startMs": 189560,
    "text": "our code review surface areas, GitHub,"
  },
  {
    "startMs": 192120,
    "text": "Fabricator, and the agent loop."
  },
  {
    "startMs": 194800,
    "text": "These all feed into you review service."
  },
  {
    "startMs": 198760,
    "text": "These This takes in requests for"
  },
  {
    "startMs": 200280,
    "text": "reviews."
  },
  {
    "startMs": 201519,
    "text": "It brings in feedback from users, and it"
  },
  {
    "startMs": 204239,
    "text": "routes it. We have a number of different"
  },
  {
    "startMs": 205920,
    "text": "generators. Now, these generators are"
  },
  {
    "startMs": 208440,
    "text": "tuned for different performance and cost"
  },
  {
    "startMs": 213160,
    "text": "avenues. There are We also have the"
  },
  {
    "startMs": 215160,
    "text": "ability to plug into third-party code"
  },
  {
    "startMs": 217840,
    "text": "review systems so that we can compare"
  },
  {
    "startMs": 219800,
    "text": "ourselves to what's available more"
  },
  {
    "startMs": 222120,
    "text": "broadly."
  },
  {
    "startMs": 223239,
    "text": "Finally, with all these different"
  },
  {
    "startMs": 224360,
    "text": "generators, we might be might be"
  },
  {
    "startMs": 226320,
    "text": "duplicating comments, and we can"
  },
  {
    "startMs": 228560,
    "text": "actually create quite a high volume of"
  },
  {
    "startMs": 231120,
    "text": "comments. If you've ever used AI to to"
  },
  {
    "startMs": 233640,
    "text": "run a code review, you've probably seen"
  },
  {
    "startMs": 234840,
    "text": "that. So, we run through a number of"
  },
  {
    "startMs": 237800,
    "text": "steps in the post-processing where we"
  },
  {
    "startMs": 240480,
    "text": "both rate, categorize, filter, and"
  },
  {
    "startMs": 243000,
    "text": "deduplicate comments so that our"
  },
  {
    "startMs": 244640,
    "text": "engineers get only the highest"
  },
  {
    "startMs": 246880,
    "text": "confidence comments that are actionable"
  },
  {
    "startMs": 248880,
    "text": "for them to work on."
  },
  {
    "startMs": 250880,
    "text": "You'll also notice along the bottom we"
  },
  {
    "startMs": 252640,
    "text": "talk a little bit about feedback in our"
  },
  {
    "startMs": 254400,
    "text": "evaluation. But, with this context of"
  },
  {
    "startMs": 256560,
    "text": "the overall system, I'm now going to"
  },
  {
    "startMs": 258239,
    "text": "hand it off to Ameya to dive into our"
  },
  {
    "startMs": 259920,
    "text": "first focus area."
  },
  {
    "startMs": 264200,
    "text": ">> Hello."
  },
  {
    "startMs": 265200,
    "text": "Hello, everyone. So, I will be talking"
  },
  {
    "startMs": 267440,
    "text": "about how we evolve U review with"
  },
  {
    "startMs": 270200,
    "text": "observability and evaluation."
  },
  {
    "startMs": 273520,
    "text": "So, U review had a very humble"
  },
  {
    "startMs": 274880,
    "text": "beginning. Basically, it was a single"
  },
  {
    "startMs": 277200,
    "text": "prompt that you should do logic checks"
  },
  {
    "startMs": 278840,
    "text": "per file, a simple agent which used to"
  },
  {
    "startMs": 280919,
    "text": "do thorough review. And we had a"
  },
  {
    "startMs": 282960,
    "text": "dispatcher to decide whether to go which"
  },
  {
    "startMs": 286200,
    "text": "generator to choose."
  },
  {
    "startMs": 288000,
    "text": "Even what we used to collect as"
  },
  {
    "startMs": 289360,
    "text": "observability was very surface-level. We"
  },
  {
    "startMs": 291919,
    "text": "used to collect cost. We used to run an"
  },
  {
    "startMs": 293880,
    "text": "NPS survey, have Google Forms being"
  },
  {
    "startMs": 295960,
    "text": "filled, Slack support. And with all of"
  },
  {
    "startMs": 298600,
    "text": "this, we saw that our quality to cost"
  },
  {
    "startMs": 300680,
    "text": "ratio was like all over the place. Like,"
  },
  {
    "startMs": 303480,
    "text": "our goal is to be in the second"
  },
  {
    "startMs": 304919,
    "text": "quadrant, that is the top left quadrant,"
  },
  {
    "startMs": 307840,
    "text": "but"
  },
  {
    "startMs": 308840,
    "text": "you can see we were all over the place."
  },
  {
    "startMs": 311200,
    "text": "Then what we did is that we started"
  },
  {
    "startMs": 313280,
    "text": "collecting more data. So, we started"
  },
  {
    "startMs": 317640,
    "text": "collecting the sentiments of the replies"
  },
  {
    "startMs": 320160,
    "text": "that were made to the U review"
  },
  {
    "startMs": 322240,
    "text": "that the U review uh call"
  },
  {
    "startMs": 325040,
    "text": "you know, the U review agent got from"
  },
  {
    "startMs": 327160,
    "text": "the developers. So, we categorized them"
  },
  {
    "startMs": 329280,
    "text": "into positive, negative. We classified"
  },
  {
    "startMs": 331440,
    "text": "them into"
  },
  {
    "startMs": 332960,
    "text": "various categories, and we found a bunch"
  },
  {
    "startMs": 335240,
    "text": "a lot of classes of bugs and issues that"
  },
  {
    "startMs": 337280,
    "text": "we could actually solve. And with that,"
  },
  {
    "startMs": 340240,
    "text": "we improved the system, and we were able"
  },
  {
    "startMs": 342240,
    "text": "to move a large number of PRs to a high"
  },
  {
    "startMs": 345440,
    "text": "quality to cost ratio."
  },
  {
    "startMs": 347560,
    "text": "Um but, we still felt that this was not"
  },
  {
    "startMs": 349880,
    "text": "enough."
  },
  {
    "startMs": 351240,
    "text": "We need to know more of how the review"
  },
  {
    "startMs": 353680,
    "text": "is done. So we started tracking things"
  },
  {
    "startMs": 355840,
    "text": "like address rate. So basically when a U"
  },
  {
    "startMs": 358520,
    "text": "review comment is made, does the"
  },
  {
    "startMs": 360640,
    "text": "developer go and actually address the"
  },
  {
    "startMs": 362760,
    "text": "comment? We started tracking that. And"
  },
  {
    "startMs": 365880,
    "text": "then we also"
  },
  {
    "startMs": 367320,
    "text": "started doing more like a runtime"
  },
  {
    "startMs": 368920,
    "text": "profile, which is like the agent"
  },
  {
    "startMs": 370720,
    "text": "trajectory,"
  },
  {
    "startMs": 371960,
    "text": "which told us"
  },
  {
    "startMs": 373520,
    "text": "why the agent is doing what it what it"
  },
  {
    "startMs": 375960,
    "text": "did. We get to know what tools calls it"
  },
  {
    "startMs": 379400,
    "text": "made. We get to know what thinking"
  },
  {
    "startMs": 380919,
    "text": "process it had. And then with that"
  },
  {
    "startMs": 382919,
    "text": "insight, we were able to actually tune"
  },
  {
    "startMs": 385120,
    "text": "our runtime, tune our performance such"
  },
  {
    "startMs": 387520,
    "text": "that the agent could very quickly give"
  },
  {
    "startMs": 390560,
    "text": "us"
  },
  {
    "startMs": 392000,
    "text": "high-quality results at a low cost."
  },
  {
    "startMs": 395520,
    "text": "One of the biggest learnings in this"
  },
  {
    "startMs": 397200,
    "text": "process was like the model doesn't know"
  },
  {
    "startMs": 399160,
    "text": "that it's wrong. It always confidently"
  },
  {
    "startMs": 401200,
    "text": "says 100% sure that yeah, this is the"
  },
  {
    "startMs": 404160,
    "text": "review for your code. Go ahead. But we"
  },
  {
    "startMs": 406240,
    "text": "saw that no, it actually needs a lot of"
  },
  {
    "startMs": 408680,
    "text": "guidance from the teams because each"
  },
  {
    "startMs": 410800,
    "text": "team has its own style guide, its own"
  },
  {
    "startMs": 414080,
    "text": "patterns or like anti-patterns that they"
  },
  {
    "startMs": 416320,
    "text": "want to look for. So that all should be"
  },
  {
    "startMs": 418480,
    "text": "like baked into the agent. And we also"
  },
  {
    "startMs": 421160,
    "text": "realized that we need to have guardrails"
  },
  {
    "startMs": 423600,
    "text": "for the agent. So we need to tell the"
  },
  {
    "startMs": 425640,
    "text": "agent what not to waste turns doing."
  },
  {
    "startMs": 427640,
    "text": "Like code review is something that has"
  },
  {
    "startMs": 429919,
    "text": "to happen in like a specific time span."
  },
  {
    "startMs": 432919,
    "text": "And then if it starts spending time"
  },
  {
    "startMs": 434680,
    "text": "doing things that it should not be"
  },
  {
    "startMs": 436280,
    "text": "doing, uh leads to a bad quality code"
  },
  {
    "startMs": 439600,
    "text": "review."
  },
  {
    "startMs": 440760,
    "text": "Uh second focus area for U review has"
  },
  {
    "startMs": 444000,
    "text": "been"
  },
  {
    "startMs": 445000,
    "text": "nations."
  },
  {
    "startMs": 446160,
    "text": "We"
  },
  {
    "startMs": 447240,
    "text": "We went very deep on team customizations"
  },
  {
    "startMs": 449400,
    "text": "because as we'll presented that we have"
  },
  {
    "startMs": 451240,
    "text": "hundreds of teams and everyone has like"
  },
  {
    "startMs": 453440,
    "text": "their own way or their own thing for"
  },
  {
    "startMs": 455680,
    "text": "code review."
  },
  {
    "startMs": 457280,
    "text": "So our review stack is pretty"
  },
  {
    "startMs": 458600,
    "text": "straightforward. We have single-file"
  },
  {
    "startMs": 460600,
    "text": "reviewers and multi-file reviewers."
  },
  {
    "startMs": 463200,
    "text": "Uh, we basically do a general purpose"
  },
  {
    "startMs": 465680,
    "text": "\"Hey, find me all logic bugs per file\""
  },
  {
    "startMs": 468160,
    "text": "uh, kind of a review. And uh, then we"
  },
  {
    "startMs": 470680,
    "text": "also do a deep review because we have"
  },
  {
    "startMs": 472520,
    "text": "like six mono repos. So, all these mono"
  },
  {
    "startMs": 474960,
    "text": "repos have their own anti-pattern style"
  },
  {
    "startMs": 477120,
    "text": "guides and all baked into this agent"
  },
  {
    "startMs": 479040,
    "text": "review which does have a nice multi-file"
  },
  {
    "startMs": 481400,
    "text": "review."
  },
  {
    "startMs": 482400,
    "text": "But, then we extended it further"
  },
  {
    "startMs": 484920,
    "text": "uh, basically to AI linters. These are"
  },
  {
    "startMs": 487120,
    "text": "basically few shot uh,"
  },
  {
    "startMs": 489440,
    "text": "AI problem uh, or like a few shots uh,"
  },
  {
    "startMs": 492360,
    "text": "system where uh, developers can"
  },
  {
    "startMs": 495040,
    "text": "basically"
  },
  {
    "startMs": 496560,
    "text": "kind of deterministically get more"
  },
  {
    "startMs": 498440,
    "text": "context and then run rules with that"
  },
  {
    "startMs": 501200,
    "text": "context and like a file and find some"
  },
  {
    "startMs": 504360,
    "text": "uh, systematic and mechanical issues."
  },
  {
    "startMs": 506800,
    "text": "And finally uh, the most powerful thing"
  },
  {
    "startMs": 509160,
    "text": "is the custom agent uh, where the teams"
  },
  {
    "startMs": 511880,
    "text": "could basically define their own custom"
  },
  {
    "startMs": 513680,
    "text": "agent, link it to like a knowledge base,"
  },
  {
    "startMs": 516120,
    "text": "uh, link it to their past PRs, have like"
  },
  {
    "startMs": 518560,
    "text": "a skill to do the review, and so on."
  },
  {
    "startMs": 521919,
    "text": "But, uh, all of this was not simple"
  },
  {
    "startMs": 524000,
    "text": "because we had to actually uh, piggyback"
  },
  {
    "startMs": 526480,
    "text": "on our uh, ownership model which is at"
  },
  {
    "startMs": 528520,
    "text": "Uber uh, so that we can like very"
  },
  {
    "startMs": 530560,
    "text": "logically roll out to all the teams."
  },
  {
    "startMs": 534120,
    "text": "Uh, we had to basically do a"
  },
  {
    "startMs": 537280,
    "text": "uh, what do you say? Co-locate the"
  },
  {
    "startMs": 538720,
    "text": "customizations next to where the"
  },
  {
    "startMs": 540480,
    "text": "developers write their code so that they"
  },
  {
    "startMs": 542280,
    "text": "can like quickly uh, keep updating these"
  },
  {
    "startMs": 544560,
    "text": "customizations. We had to implement a"
  },
  {
    "startMs": 546960,
    "text": "smart deterministic uh, routing so that"
  },
  {
    "startMs": 550400,
    "text": "we could route which team gets what kind"
  },
  {
    "startMs": 552480,
    "text": "of review with which model, what kind of"
  },
  {
    "startMs": 554600,
    "text": "generators, and so on."
  },
  {
    "startMs": 556480,
    "text": "And finally uh, the hard thing was like"
  },
  {
    "startMs": 558720,
    "text": "we had to actually surface all of this"
  },
  {
    "startMs": 560640,
    "text": "observability that I talked before, like"
  },
  {
    "startMs": 563040,
    "text": "the agent trajectory, addressal rate,"
  },
  {
    "startMs": 565520,
    "text": "uh, sentiment analysis back to the"
  },
  {
    "startMs": 567320,
    "text": "teams. So, so that the teams could"
  },
  {
    "startMs": 569240,
    "text": "actually understand that \"Oh, I wrote"
  },
  {
    "startMs": 571320,
    "text": "this rule, but maybe not a lot of"
  },
  {
    "startMs": 573880,
    "text": "developers are liking it in my team, so"
  },
  {
    "startMs": 575640,
    "text": "let me go and update it.\" And then we"
  },
  {
    "startMs": 577520,
    "text": "had to give Bubble up that kind of"
  },
  {
    "startMs": 579120,
    "text": "observability to all the people who are"
  },
  {
    "startMs": 581520,
    "text": "contributing to the platform."
  },
  {
    "startMs": 583680,
    "text": "Uh"
  },
  {
    "startMs": 584480,
    "text": "one thing that we learned is that"
  },
  {
    "startMs": 586040,
    "text": "actually writing the skill was very"
  },
  {
    "startMs": 588560,
    "text": "easy. Like teams just very quickly wrote"
  },
  {
    "startMs": 591400,
    "text": "a skill by asking Claude to write one,"
  },
  {
    "startMs": 593720,
    "text": "go over my"
  },
  {
    "startMs": 595080,
    "text": "previous PR reviews and write me a"
  },
  {
    "startMs": 597240,
    "text": "skill. But the hard part was how to run"
  },
  {
    "startMs": 600320,
    "text": "these skills at scale with consistent"
  },
  {
    "startMs": 602680,
    "text": "quality and low cost. And that required"
  },
  {
    "startMs": 605120,
    "text": "a lot of iterations not only from the"
  },
  {
    "startMs": 607160,
    "text": "U-Review team side, but also like for"
  },
  {
    "startMs": 609520,
    "text": "each team who was trying to write these"
  },
  {
    "startMs": 611200,
    "text": "rules. Uh"
  },
  {
    "startMs": 613960,
    "text": "in results, we basically uh see that,"
  },
  {
    "startMs": 616520,
    "text": "you know, uh U-Review does like around"
  },
  {
    "startMs": 618520,
    "text": "25,000 comments a week. And uh we get"
  },
  {
    "startMs": 621880,
    "text": "10% of them actually get some feedback."
  },
  {
    "startMs": 624480,
    "text": "And only 4% of the PRs actually get some"
  },
  {
    "startMs": 626800,
    "text": "negative feedback. Uh we also saw that"
  },
  {
    "startMs": 630000,
    "text": "um"
  },
  {
    "startMs": 631040,
    "text": "the overall addressal rate was uh around"
  },
  {
    "startMs": 633200,
    "text": "67%"
  },
  {
    "startMs": 634760,
    "text": "and almost three quarters of the high"
  },
  {
    "startMs": 636600,
    "text": "severity issues uh"
  },
  {
    "startMs": 639560,
    "text": "were usually addressed by the"
  },
  {
    "startMs": 640800,
    "text": "developers, which shows that U-Review"
  },
  {
    "startMs": 642600,
    "text": "actually adds some value to the entire"
  },
  {
    "startMs": 645000,
    "text": "development life cycle. And then uh with"
  },
  {
    "startMs": 647680,
    "text": "all the observability and"
  },
  {
    "startMs": 649760,
    "text": "uh evals that I showed that I went"
  },
  {
    "startMs": 651640,
    "text": "through, we saw that against like a very"
  },
  {
    "startMs": 653480,
    "text": "naive implementation, our costs were"
  },
  {
    "startMs": 655880,
    "text": "down by 60% and our quality and our"
  },
  {
    "startMs": 658600,
    "text": "accuracy was up by uh around 70%."
  },
  {
    "startMs": 662160,
    "text": "Uh"
  },
  {
    "startMs": 662760,
    "text": "for a last focus area, I'll give the mic"
  },
  {
    "startMs": 665000,
    "text": "back to Will and he will go over the"
  },
  {
    "startMs": 667160,
    "text": "inner versus outer loop."
  },
  {
    "startMs": 671600,
    "text": ">> Awesome. So, now that we've talked about"
  },
  {
    "startMs": 674560,
    "text": "uh some of the details of actually"
  },
  {
    "startMs": 676640,
    "text": "implementing high-quality reviews, it"
  },
  {
    "startMs": 679000,
    "text": "kind of brings us to the last area,"
  },
  {
    "startMs": 681480,
    "text": "which is where we start talking about"
  },
  {
    "startMs": 684080,
    "text": "where things are going, right? With"
  },
  {
    "startMs": 685839,
    "text": "moving to the Agentech SDLC,"
  },
  {
    "startMs": 688880,
    "text": "we're moving software into a model where"
  },
  {
    "startMs": 691800,
    "text": "engineers are interacting with the code"
  },
  {
    "startMs": 694240,
    "text": "less."
  },
  {
    "startMs": 695400,
    "text": "They're often times not as involved in"
  },
  {
    "startMs": 697320,
    "text": "authoring the code. Uh currently, we"
  },
  {
    "startMs": 700000,
    "text": "still have uh humans approving the code,"
  },
  {
    "startMs": 703280,
    "text": "uh but we see a a short path in the near"
  },
  {
    "startMs": 706920,
    "text": "future to a percentage of our code"
  },
  {
    "startMs": 710040,
    "text": "landing automatically, having automatic"
  },
  {
    "startMs": 711920,
    "text": "approvals, right? The various parts of"
  },
  {
    "startMs": 714160,
    "text": "the industry are already moving there."
  },
  {
    "startMs": 716400,
    "text": "Um"
  },
  {
    "startMs": 717280,
    "text": "Part of the way along the process was"
  },
  {
    "startMs": 719800,
    "text": "figuring out by having our single code"
  },
  {
    "startMs": 722800,
    "text": "review platform, what did we need to"
  },
  {
    "startMs": 724920,
    "text": "tune for the various audiences that are"
  },
  {
    "startMs": 727880,
    "text": "actually getting these code reviews? Um"
  },
  {
    "startMs": 730560,
    "text": "you know, the interface, that's one area"
  },
  {
    "startMs": 732360,
    "text": "that's sort of intuitive there. Uh one"
  },
  {
    "startMs": 734840,
    "text": "thing that might be less intuitive is"
  },
  {
    "startMs": 736680,
    "text": "around accuracy. Uh with the inner loop,"
  },
  {
    "startMs": 740880,
    "text": "our accuracy needs actually need to go"
  },
  {
    "startMs": 742640,
    "text": "up,"
  },
  {
    "startMs": 743600,
    "text": "or else we can result in uh dealing with"
  },
  {
    "startMs": 746480,
    "text": "cavitation of an agent where it fixes"
  },
  {
    "startMs": 748160,
    "text": "something, goes back, gets another code"
  },
  {
    "startMs": 749560,
    "text": "review, and has to kind of like fix"
  },
  {
    "startMs": 751360,
    "text": "backwards because the quality of the"
  },
  {
    "startMs": 753120,
    "text": "comment was low. Um"
  },
  {
    "startMs": 756480,
    "text": "The one of the other interesting things"
  },
  {
    "startMs": 758160,
    "text": "is agents are more than happy to go"
  },
  {
    "startMs": 759920,
    "text": "through and fix 100 nits on a pull"
  },
  {
    "startMs": 762400,
    "text": "request where your engineers really get"
  },
  {
    "startMs": 764680,
    "text": "frustrated in situations like that. Um"
  },
  {
    "startMs": 767720,
    "text": "but probably the most interesting aspect"
  },
  {
    "startMs": 769920,
    "text": "of this transition is the feedback. As"
  },
  {
    "startMs": 773400,
    "text": "you can see, quite a bit of what went"
  },
  {
    "startMs": 775600,
    "text": "into getting high-quality code reviews"
  },
  {
    "startMs": 777960,
    "text": "at Uber was bringing the human feedback"
  },
  {
    "startMs": 781960,
    "text": "into the system and using that to figure"
  },
  {
    "startMs": 784640,
    "text": "out how to tune our prompts, how to tune"
  },
  {
    "startMs": 786839,
    "text": "our agents."
  },
  {
    "startMs": 788080,
    "text": "Uh and so as we move to a model where"
  },
  {
    "startMs": 792320,
    "text": "humans are less in the loop, where"
  },
  {
    "startMs": 795120,
    "text": "software engineering is moving to an"
  },
  {
    "startMs": 797000,
    "text": "agentic model,"
  },
  {
    "startMs": 799600,
    "text": "we're effectively going to a place where"
  },
  {
    "startMs": 801280,
    "text": "we're starting to talk about are we"
  },
  {
    "startMs": 803000,
    "text": "going to kill the outer loop? Is the"
  },
  {
    "startMs": 804839,
    "text": "human engineer not going to be involved"
  },
  {
    "startMs": 808120,
    "text": "in the code review."
  },
  {
    "startMs": 809400,
    "text": "Some people are already here."
  },
  {
    "startMs": 811560,
    "text": "Now,"
  },
  {
    "startMs": 812800,
    "text": "with the feedback taken into"
  },
  {
    "startMs": 814200,
    "text": "consideration,"
  },
  {
    "startMs": 815840,
    "text": "you start wondering, all right,"
  },
  {
    "startMs": 817720,
    "text": "what could this result in, right? I'll"
  },
  {
    "startMs": 819360,
    "text": "let your imagination go there in terms"
  },
  {
    "startMs": 821280,
    "text": "of quality degradation, slop, and so"
  },
  {
    "startMs": 823400,
    "text": "forth. But, rather than killing the"
  },
  {
    "startMs": 825360,
    "text": "outer loop, I think that we believe and"
  },
  {
    "startMs": 828560,
    "text": "the industry has just started to really"
  },
  {
    "startMs": 830520,
    "text": "kind of coalesce on this idea that we're"
  },
  {
    "startMs": 832960,
    "text": "really expanding the outer loop. Rather"
  },
  {
    "startMs": 835400,
    "text": "than removing humans from the code"
  },
  {
    "startMs": 837800,
    "text": "review process, we are moving their"
  },
  {
    "startMs": 841160,
    "text": "responsibilities up a layer."
  },
  {
    "startMs": 843560,
    "text": "Rather than them dealing with the"
  },
  {
    "startMs": 845040,
    "text": "details of the implementation, the agent"
  },
  {
    "startMs": 847160,
    "text": "is great at writing the software."
  },
  {
    "startMs": 849400,
    "text": "The agent is getting much, much better"
  },
  {
    "startMs": 851400,
    "text": "at reviewing the software as a human"
  },
  {
    "startMs": 853360,
    "text": "would. But now, as software engineers,"
  },
  {
    "startMs": 855960,
    "text": "we still are going to have an outer"
  },
  {
    "startMs": 857200,
    "text": "loop. It's just going to look a little"
  },
  {
    "startMs": 858520,
    "text": "different. Instead of you worrying about"
  },
  {
    "startMs": 860800,
    "text": "the optimization of the performance and"
  },
  {
    "startMs": 863440,
    "text": "the API compatibility, you're going to"
  },
  {
    "startMs": 865480,
    "text": "be thinking more about architecture in"
  },
  {
    "startMs": 867480,
    "text": "your code reviews. You're going to have"
  },
  {
    "startMs": 869080,
    "text": "time to focus on the domain expertise"
  },
  {
    "startMs": 871600,
    "text": "that you have and product thinking. So,"
  },
  {
    "startMs": 874880,
    "text": "we believe that as we adopt this"
  },
  {
    "startMs": 876960,
    "text": "automated uh code review, this is going"
  },
  {
    "startMs": 880080,
    "text": "to be the result of how our engineers"
  },
  {
    "startMs": 882360,
    "text": "are interacting with the system and"
  },
  {
    "startMs": 883720,
    "text": "guiding it."
  },
  {
    "startMs": 887360,
    "text": "And that's it. Thank you so much for"
  },
  {
    "startMs": 888680,
    "text": "coming. Thanks."
  }
]
