# How One Founder Manages 20 AI Coding Agents at Once - Ryan Carson, Untangle

Ryan Carson, Untangle

[Talk page](https://ai.engineer/talks/BTtyguoPvBU-one-founder-manages-20-ai-coding-agents) · [Video](https://www.youtube.com/watch?v=BTtyguoPvBU)

## How One Founder Manages 20 AI Coding Agents at Once

At Untangle, Ryan Carson directs about 20 builders through one conversation with an engineering manager agent. When a feature is ready, the manager sends him a video. That is where he slows down.

### From a customer priority to one pull request

[0:12](https://www.youtube.com/watch?v=BTtyguoPvBU&t=12s)

[Untangle](https://untangle.us) began as a consumer app for people going through a divorce. It soon shifted to selling an AI paralegal to family law firms. Founder and CEO Ryan Carson had seen his previous startup grow to about 110 people, and one part of the management job felt familiar. Someone still had to talk to customers, decide what mattered, and turn that decision into work. At Untangle, people kept the first two responsibilities and gave a cloud agent session the job of engineering manager.

The manager leaves all the coding to child sessions called builders. Carson mostly talks to the manager by voice because he finds a spoken update easier to absorb than a written one. He can follow one conversation while several changes are underway.

A Linear ticket board tells the manager what needs building. Untangle aims to size each ticket to one pull request (PR). That gives a builder a defined change to make and the team a single PR to review. The manager uses dependencies between tickets to decide which work can start next.

### One manager checks every builder every 15 minutes

[3:29](https://www.youtube.com/watch?v=BTtyguoPvBU&t=209s)

About 20 builders can work concurrently under one manager. Carson puts the working range at roughly five to 25. The upper end depends partly on the person listening to the updates. Even if the manager can reach every builder, it can have more changes underway than its human can follow.

Carson stays out of the individual builder sessions. A builder with a technical question sends it to the manager, which can often answer it or clear a small obstacle. Those ordinary questions are precisely the ones worth delegating. If each required Carson to open a session and recover its context, he would still have 20 conversations to manage.

What makes the manager check on a builder that has gone quiet? A scheduled automation tells the manager to read every child session and move stalled work along. The interval shown in the talk is 10 minutes. By the time he presented, Carson had settled on 15. The script makes the rounds happen on schedule, something he found unreliable when left to the agent. The manager still decides what each builder needs.

![Below the manager, a Linear board assigns one builder per ticket, with 20 builders working at once. The manager checks in every 10 minutes. The reviewer sends bugs back for up to three rounds of fixes.](https://ai.engineer/api/talk-essay-media/6afc979a58f875603fa416e092b03021bebb304fb5bcd7fe9207c8ce0225a74e)

A manager distributes tickets to builders. Each ticket keeps its own agent and review assignment.

### Three review rounds and a separate QA agent

[5:04](https://www.youtube.com/watch?v=BTtyguoPvBU&t=304s)

Each builder produces code and tests. Where does it send the change for a check, and how does a finding get back to Carson? Two kinds of session branch off the builder.

- **Code review.** The builder calls a built-in reviewer and can revise the PR in response to its findings. Untangle usually caps the exchange at three rounds. At that point the builder can raise important findings with the engineering manager, which can bring in Carson if needed.
- **Quality assurance.** A separate process starts a child of the builder to run QA. This gives the change another check outside the exchange between the builder and its code reviewer.

The cost Carson watches is the delay at the merge queue. He considers a QA session cheap enough to start wherever it helps. But every extra round spent picking at a PR keeps that change waiting to land. The cap gives the builder a point at which to take an important finding to someone who can decide what to do about it.

How review, QA, and escalation connect to a builder
Only the builder and reviewer participate in the capped exchange. QA runs in a child session of its own. The engineering manager receives important findings and decides whether Carson needs to weigh in.
- Builder — Implements the ticket and writes tests.
- Code reviewer — Reviews the builder’s change.
- QA child session — Runs a separate check for the builder.
- Engineering manager — Handles findings the builder raises.
- Carson — Weighs in when the manager needs help.
- Builder → Code reviewer (Requests review)
- Code reviewer → Builder (Returns findings in each round)
- Builder → QA child session (Starts a QA child)
- Builder → Engineering manager (Can raise important findings after about three rounds)
- Engineering manager → Carson (Asks for help if needed)

### What the product owner looks for in a green test run

[6:25](https://www.youtube.com/watch?v=BTtyguoPvBU&t=385s)

What did the user actually see while the tests passed? Every builder records the feature in use. The built-in testing tool creates test cases, records the interaction, and gives each case a green or red result. For a UI change, the product owner can compare the two outputs. The results identify which checks passed. The recording shows what the feature did and how it looked while passing them.

The manager alerts Carson when a video is ready. Coming back to one PR after directing many builders, he first reads a short summary of what the change is supposed to do. That gives him a specific expectation to check as he watches. The decision to ship belongs to a person, and this is where the product owner has to give one change their full attention.

Just before the talk, Carson was speaking to the manager on his phone when it told him a video was ready. He could carry on the conversation just fine. When he played the recording, though, he could barely make out the change. Tempted to approve it anyway, he held off until he could watch on his machine. After hundreds of adjustments, Carson credited the process with helping Untangle ship a lot of high-quality PRs. It worked for his team at the time, and he expected it to change quickly.

### Many managers, one merge queue

[8:21](https://www.youtube.com/watch?v=BTtyguoPvBU&t=501s)

Carson’s advice to move development to the cloud follows from how he runs a team. Take his example of ten people working at once. Each would typically have an engineering manager session, and one person can have more than one. Carson himself had two running. The sessions need to know which other sessions exist and be able to message them directly. When one has information another needs, they can unblock each other without asking a person to relay the question.

Database migration numbers illustrate what still needs coordination across the team. Two builders working on different PRs can each reach for the same next migration number. Each number looks reasonable in its own PR. Bring the changes together and the numbers collide. Untangle assigns another session, the merge queue manager, to coordinate details like these across incoming PRs.

Like the engineering manager, the merge queue manager is an ordinary cloud session. Carson names it for its job and gives it latitude to work out how to coordinate the wave of changes. GitHub provides the merge queue itself.

### The first hiring screen is a feature PR

[11:07](https://www.youtube.com/watch?v=BTtyguoPvBU&t=667s)

Running engineering this way changes the first question Untangle asks about a candidate. Can this person direct an agent through shipping a feature? Carson had found it easy to come away from a screening call impressed by someone’s hobbies or the way they spoke. Untangle delays that conversation until the team has seen the person work.

The first submission is an unpaid, roughly 45-minute recording of the candidate’s whole screen. The assignment is to ship a feature PR with an agent, typically on an application the candidate already maintains. Untangle leaves the method up to them. The PR shows what they shipped. The recording lets a reviewer follow the directions they gave the agent and what they did with its work.

A ranking agent takes the videos and transcripts and puts the candidates in order, saving someone the time it would take to watch every submission. Carson distrusted its first ranking. In hindsight, he thought he should have hired its first choice and attributed his resistance to his own bias. That was his judgment about one hiring decision. The recordings also vary in what they reveal. Some candidates narrate freely. When someone mumbles or works silently, a reviewer may see what they did and still have little idea why.

### The top five prepare a real PR for merge

[13:23](https://www.youtube.com/watch?v=BTtyguoPvBU&t=803s)

The five highest-ranked candidates receive an invitation to a paid trial. Untangle pays a fixed fee for 16 hours, asks each candidate to sign a nondisclosure agreement, and provides a seat in its cloud coding environment. This time the task is to get a real Untangle PR ready for merge. The team gets to see how someone who could ship a feature in a familiar app handles the work they would actually be hired to do.

Loom captures the trial, and Untangle gives the recordings to an agent for a recommendation. In the last round, Carson also watched 16 hours of footage himself. He found it grueling, but it gave him a feel for the candidates. He recommends choosing finalists before calling them, so an easy conversation has less chance to sway the assessment of their work. The call can then begin with the team already inclined to hire. It still serves two purposes Carson values: a final human judgment and a chance for the two people to connect.

![Candidates record 45-minute videos of their full screen as they use agents to ship a feature in an app they maintain. A ranking agent watches every video. The top five record their screens while preparing a pull request in Untangle’s codebase for a fixed fee, with a 16-hour limit. One finalist gets the first call.](https://ai.engineer/api/talk-essay-media/2d62f104339415c1729f2d43cbc57186a897011c77bdfbd40d150b749b48cd44)

Five finalists get a real pull request to prepare. The handoff from the ranking agent marks the start of that stage.

## Key takeaways

- One PR per ticket gives the manager a unit of work to assign. Dependencies guide its next assignment, and a scheduled 15-minute pass brings its attention back to every active builder.
- The builder and reviewer usually get up to three rounds together. Important findings can then go to the manager. A separate child session handles QA.
- Carson reads what a PR is supposed to do before watching its test video. Even when every case passes, the product owner still has to judge the behavior on screen.
- Untangle evaluates agent operators first on a familiar app, then invites the top five to a paid 16-hour trial on a company PR. Carson reserves the first live call for finalists.

## Slides

### Platinum sponsors

Four white logos on black credit the platinum sponsors. From left to right, they are Oracle AI Database, Sonar, WorkOS and exa.

[0:08](https://www.youtube.com/watch?v=BTtyguoPvBU&t=8s) [Under “Platinum Sponsors,” four white logos appear on black, from left to right: Oracle AI Database, Sonar, WorkOS and exa.](https://assets.ai.engineer/sites/aiecode2025/sha256/d69b713e4b664e49ebbf0648f9b6b05cc601e68f4d66057222f5d0234fe1ef68.webp)

PLATINUM SPONSORS

ORACLE
AI Database

Sonar
Makers of SonarQube

WorkOS

exa

### Untangle’s website and the presenter’s X profile

A tangled line straightens into an arrow above Untangle’s website. Beside it, the X logo marks founder Ryan Carson’s profile.

[0:12](https://www.youtube.com/watch?v=BTtyguoPvBU&t=12s) [A tangled line straightens into an arrow above the Untangle website address. The X logo sits above the presenter’s profile link.](https://assets.ai.engineer/sites/aiecode2025/sha256/525abcd95370ac782eae63c3032558e35d567031ffe87162881e894f5a08250a.webp)

untangle.us
x.com/ryancarson

### The org chart

A human sets weekly priorities. The manager agent assigns one builder per Linear ticket, with 20 builders working at once. Each change needs tests, a QA check and a video showing it works before a human types “Ship it”.

[1:54](https://www.youtube.com/watch?v=BTtyguoPvBU&t=114s) [At the top of Untangle’s org chart, a box labeled “Human” sits between “weekly priorities” and “talks to customers.” The space below is largely blank.](https://assets.ai.engineer/sites/aiecode2025/sha256/95b56556ae6d57aadbfadd5064ea67078411110cf9ce9deb196ec6d500796253.webp)

The org chart

Human
weekly priorities
talks to customers

Untangle
AI for divorce

[3:27](https://www.youtube.com/watch?v=BTtyguoPvBU&t=207s) [Below the manager, a Linear board assigns one builder per ticket, with 20 builders working at once. The manager checks in every 10 minutes. The reviewer sends bugs back for up to three rounds of fixes.](https://assets.ai.engineer/sites/aiecode2025/sha256/6afc979a58f875603fa416e092b03021bebb304fb5bcd7fe9207c8ce0225a74e.webp)

The org chart

Human
weekly priorities
talks to customers

“what needs me?”
audio chat
1 question + recommendation

Manager agent
never writes code
checks in every 10 min
tickets

Ticket board Linear
no ticket, no work
1 per ticket

Builder agents
20 at once
questions

review
Reviewer agent
fix bugs up to 3×

Untangle
AI for divorce

[5:37](https://www.youtube.com/watch?v=BTtyguoPvBU&t=337s) [A human sets weekly priorities. A manager agent assigns Linear tickets to builder agents, who work with a reviewer. Each change proceeds through code and tests, a QA check, a video recording and a human typing “Ship it.”](https://assets.ai.engineer/sites/aiecode2025/sha256/1af633d43a442dba7fb5d86743c4c1d38876b080a3938e868ba2ac61c50ef913.webp)

The org chart

Human
weekly priorities
talks to customers

“what needs me?”
audio chat
1 question + recommendation

Manager agent
never writes code
checks in every 10 min
tickets

Ticket board Linear
no ticket, no work
1 per ticket

Builder agents
20 at once
questions

review
Reviewer agent
fix bugs up to 3×

every change
1  Code + tests
   failing test first
2  QA agent
   checks the proof
3  Video
   records it working
4  “Ship it”
   a human types it

Untangle
AI for divorce

### The team

At Untangle, 10 people each work with a manager agent. The agents coordinate with each other and hand ready pull requests to a merge manager, which queues them in GitHub so they merge into main in order.

[9:47](https://www.youtube.com/watch?v=BTtyguoPvBU&t=587s) [Ten humans each work with a manager agent. The agents coordinate and send ready work to a merge manager agent, which queues changes in GitHub for merging into main in order.](https://assets.ai.engineer/sites/aiecode2025/sha256/5cfccd4eb4a001e8cf5a12b93c48ee363db116d098cc0fb2aaaa1ad521ef6415.webp)

The team

10 humans
10 manager agents
coordinate

ready to merge
Merge manager agent
queues
GitHub merge queue
in order
main

Untangle
AI for divorce

Interface text:
+

### How we hire

An agent ranks 45-minute videos of candidates using agents to ship a feature in an app they maintain. The top five get a fixed fee and up to 16 hours to prepare a pull request in Untangle’s codebase. Only the finalist gets a call.

[14:04](https://www.youtube.com/watch?v=BTtyguoPvBU&t=844s) [Candidates record 45-minute videos of their full screen as they use agents to ship a feature in an app they maintain. A ranking agent watches every video. The top five record their screens while preparing a pull request in Untangle’s codebase for a fixed fee, with a 16-hour limit. One finalist gets the first call.](https://assets.ai.engineer/sites/aiecode2025/sha256/2d62f104339415c1729f2d43cbc57186a897011c77bdfbd40d150b749b48cd44.webp)

How we hire

Untangle
AI for divorce

Candidates
45-min whole-screen video
agents ship new feature
app they maintain

videos

Ranking agent
watches every video

invite top 5

Top 5
real PR, our codebase
whole screen recorded
fixed fee
max 16 hours

pick 1

Finalist

no interview calls
first call

## Full transcript

[0:01](https://www.youtube.com/watch?v=BTtyguoPvBU&t=1s) [music]

[0:12](https://www.youtube.com/watch?v=BTtyguoPvBU&t=12s) So, my name is Ryan Carson. I'm the

[0:14](https://www.youtube.com/watch?v=BTtyguoPvBU&t=14s) founder and CEO of Untangle. So, we're

[0:18](https://www.youtube.com/watch?v=BTtyguoPvBU&t=18s) building an AI parallegal specifically

[0:20](https://www.youtube.com/watch?v=BTtyguoPvBU&t=20s) for family law, uh, focusing on divorce

[0:23](https://www.youtube.com/watch?v=BTtyguoPvBU&t=23s) in particular. Um, so, quick audience

[0:25](https://www.youtube.com/watch?v=BTtyguoPvBU&t=25s) poll. Who has been divorced? Raise your

[0:27](https://www.youtube.com/watch?v=BTtyguoPvBU&t=27s) hand. Raise it. Be don't be shy. Raise

[0:29](https://www.youtube.com/watch?v=BTtyguoPvBU&t=29s) your hand. Who's been divorced? Okay,

[0:30](https://www.youtube.com/watch?v=BTtyguoPvBU&t=30s) keep your hands up. Who knows somebody

[0:32](https://www.youtube.com/watch?v=BTtyguoPvBU&t=32s) that's been divorced? Raise your hand.

[0:34](https://www.youtube.com/watch?v=BTtyguoPvBU&t=34s) Everybody. Yeah. So, for you watching

[0:36](https://www.youtube.com/watch?v=BTtyguoPvBU&t=36s) the video, everyone raised their hand by

[0:37](https://www.youtube.com/watch?v=BTtyguoPvBU&t=37s) the end. Divorce touches a lot of

[0:39](https://www.youtube.com/watch?v=BTtyguoPvBU&t=39s) people. Um, two of my sisters got

[0:41](https://www.youtube.com/watch?v=BTtyguoPvBU&t=41s) divorced. My parents got divorced. Um,

[0:43](https://www.youtube.com/watch?v=BTtyguoPvBU&t=43s) and it's one of the most painful things

[0:45](https://www.youtube.com/watch?v=BTtyguoPvBU&t=45s) in your whole life. And I wanted to work

[0:47](https://www.youtube.com/watch?v=BTtyguoPvBU&t=47s) on that problem and just make it a

[0:49](https://www.youtube.com/watch?v=BTtyguoPvBU&t=49s) little less painful. Um, started off as

[0:51](https://www.youtube.com/watch?v=BTtyguoPvBU&t=51s) a consumer app. Uh, quickly pivoted to

[0:53](https://www.youtube.com/watch?v=BTtyguoPvBU&t=53s) B2B and we sell to family law firms now.

[0:56](https://www.youtube.com/watch?v=BTtyguoPvBU&t=56s) And thankfully product market fit is

[0:58](https://www.youtube.com/watch?v=BTtyguoPvBU&t=58s) there and we're growing really fast. So

[1:00](https://www.youtube.com/watch?v=BTtyguoPvBU&t=60s) that's really fun. Um so I've built and

[1:04](https://www.youtube.com/watch?v=BTtyguoPvBU&t=64s) sold four companies. Um my last startup

[1:06](https://www.youtube.com/watch?v=BTtyguoPvBU&t=66s) we scaled to about 110 people and I

[1:09](https://www.youtube.com/watch?v=BTtyguoPvBU&t=69s) learned a lot about managing people and

[1:11](https://www.youtube.com/watch?v=BTtyguoPvBU&t=71s) it turns out managing people is like

[1:13](https://www.youtube.com/watch?v=BTtyguoPvBU&t=73s) managing agents. And so I'm going to

[1:14](https://www.youtube.com/watch?v=BTtyguoPvBU&t=74s) share with you how we manage agents at

[1:17](https://www.youtube.com/watch?v=BTtyguoPvBU&t=77s) Untangle and how everything I've learned

[1:19](https://www.youtube.com/watch?v=BTtyguoPvBU&t=79s) uh managing people kind of fits there.

[1:22](https://www.youtube.com/watch?v=BTtyguoPvBU&t=82s) So I'm hoping to teach you some of those

[1:23](https://www.youtube.com/watch?v=BTtyguoPvBU&t=83s) things. Um, I'm also going to show you

[1:26](https://www.youtube.com/watch?v=BTtyguoPvBU&t=86s) how we hire. I I think hiring is very

[1:28](https://www.youtube.com/watch?v=BTtyguoPvBU&t=88s) different in the age of agents. Um, I've

[1:30](https://www.youtube.com/watch?v=BTtyguoPvBU&t=90s) hired a lot of people over my career.

[1:32](https://www.youtube.com/watch?v=BTtyguoPvBU&t=92s) Um, I've made a lot of mistakes and I'm

[1:34](https://www.youtube.com/watch?v=BTtyguoPvBU&t=94s) going to share how we're hiring now at

[1:35](https://www.youtube.com/watch?v=BTtyguoPvBU&t=95s) Untangle and I think how it's really

[1:37](https://www.youtube.com/watch?v=BTtyguoPvBU&t=97s) working. Um, and then I'm going to show

[1:38](https://www.youtube.com/watch?v=BTtyguoPvBU&t=98s) you a live demo.

[1:41](https://www.youtube.com/watch?v=BTtyguoPvBU&t=101s) So, let's get started. This is me on X

[1:45](https://www.youtube.com/watch?v=BTtyguoPvBU&t=105s) and this is Untangle. If you want to

[1:46](https://www.youtube.com/watch?v=BTtyguoPvBU&t=106s) know more about us,

[1:49](https://www.youtube.com/watch?v=BTtyguoPvBU&t=109s) if my little clicker works, you can do

[1:51](https://www.youtube.com/watch?v=BTtyguoPvBU&t=111s) it.

[1:55](https://www.youtube.com/watch?v=BTtyguoPvBU&t=115s) Okay, so what does the org chart look

[1:58](https://www.youtube.com/watch?v=BTtyguoPvBU&t=118s) like now in the age of agents? How do

[2:01](https://www.youtube.com/watch?v=BTtyguoPvBU&t=121s) you run your agents? Um, so the way we

[2:04](https://www.youtube.com/watch?v=BTtyguoPvBU&t=124s) do it is pretty straightforward. So we

[2:05](https://www.youtube.com/watch?v=BTtyguoPvBU&t=125s) have humans at the top. Um, and

[2:07](https://www.youtube.com/watch?v=BTtyguoPvBU&t=127s) obviously we set priorities, we talk to

[2:09](https://www.youtube.com/watch?v=BTtyguoPvBU&t=129s) our customers, we're doing all the human

[2:10](https://www.youtube.com/watch?v=BTtyguoPvBU&t=130s) stuff, but what we use are manager

[2:13](https://www.youtube.com/watch?v=BTtyguoPvBU&t=133s) agents and these are literally just

[2:14](https://www.youtube.com/watch?v=BTtyguoPvBU&t=134s) sessions. So we use Devon. Love Devon.

[2:17](https://www.youtube.com/watch?v=BTtyguoPvBU&t=137s) We do all our engine the cloud. Um, I've

[2:20](https://www.youtube.com/watch?v=BTtyguoPvBU&t=140s) done [clears throat] everything, you

[2:21](https://www.youtube.com/watch?v=BTtyguoPvBU&t=141s) know, from coding by hand to, you know,

[2:24](https://www.youtube.com/watch?v=BTtyguoPvBU&t=144s) using cursor to here and we're in Devon.

[2:26](https://www.youtube.com/watch?v=BTtyguoPvBU&t=146s) But we have a manager session. I'm going

[2:28](https://www.youtube.com/watch?v=BTtyguoPvBU&t=148s) to show you this. It's literally just a

[2:29](https://www.youtube.com/watch?v=BTtyguoPvBU&t=149s) session. I I call it engineer manager.

[2:32](https://www.youtube.com/watch?v=BTtyguoPvBU&t=152s) Um, and it never writes code. And I

[2:34](https://www.youtube.com/watch?v=BTtyguoPvBU&t=154s) mostly talk to this agent. Um, and it

[2:37](https://www.youtube.com/watch?v=BTtyguoPvBU&t=157s) manages a bunch of children. So, I'll

[2:38](https://www.youtube.com/watch?v=BTtyguoPvBU&t=158s) talk to you how about how we do that. We

[2:40](https://www.youtube.com/watch?v=BTtyguoPvBU&t=160s) also I mostly use audio to manage this

[2:43](https://www.youtube.com/watch?v=BTtyguoPvBU&t=163s) agent. Um, I don't know about you, but

[2:46](https://www.youtube.com/watch?v=BTtyguoPvBU&t=166s) I'm not a fast reader. I'm definitely

[2:48](https://www.youtube.com/watch?v=BTtyguoPvBU&t=168s) not an agent and I prefer audio. I can

[2:50](https://www.youtube.com/watch?v=BTtyguoPvBU&t=170s) ingest audio a lot better for some

[2:53](https://www.youtube.com/watch?v=BTtyguoPvBU&t=173s) reason than reading. So, I talk mostly

[2:55](https://www.youtube.com/watch?v=BTtyguoPvBU&t=175s) to my manager agent over audio. I'll

[2:58](https://www.youtube.com/watch?v=BTtyguoPvBU&t=178s) show you that. Um, I just find it I I

[3:01](https://www.youtube.com/watch?v=BTtyguoPvBU&t=181s) can absorb it. I can understand it. I

[3:03](https://www.youtube.com/watch?v=BTtyguoPvBU&t=183s) can see what's happening. It's much more

[3:04](https://www.youtube.com/watch?v=BTtyguoPvBU&t=184s) natural. So, I'm going to show you that

[3:06](https://www.youtube.com/watch?v=BTtyguoPvBU&t=186s) as well. Now, the manager agent uh

[3:09](https://www.youtube.com/watch?v=BTtyguoPvBU&t=189s) primarily works from a ticket board. We

[3:11](https://www.youtube.com/watch?v=BTtyguoPvBU&t=191s) use linear. Doesn't really matter what

[3:13](https://www.youtube.com/watch?v=BTtyguoPvBU&t=193s) you use, but this is kind of the source

[3:14](https://www.youtube.com/watch?v=BTtyguoPvBU&t=194s) of truth about what needs to be built.

[3:16](https://www.youtube.com/watch?v=BTtyguoPvBU&t=196s) every ticket should be one PR um which

[3:19](https://www.youtube.com/watch?v=BTtyguoPvBU&t=199s) is how we we work and that the uh

[3:21](https://www.youtube.com/watch?v=BTtyguoPvBU&t=201s) manager agent knows which tickets to

[3:23](https://www.youtube.com/watch?v=BTtyguoPvBU&t=203s) pull based off dependencies. It's like

[3:25](https://www.youtube.com/watch?v=BTtyguoPvBU&t=205s) pretty basic engineering manager stuff

[3:27](https://www.youtube.com/watch?v=BTtyguoPvBU&t=207s) here, but this is where it gets

[3:29](https://www.youtube.com/watch?v=BTtyguoPvBU&t=209s) interesting. So each manager agent uh

[3:32](https://www.youtube.com/watch?v=BTtyguoPvBU&t=212s) concurrently manages up to about 20

[3:35](https://www.youtube.com/watch?v=BTtyguoPvBU&t=215s) builder agents. Um this number varies

[3:38](https://www.youtube.com/watch?v=BTtyguoPvBU&t=218s) like it can go anywhere from five to

[3:40](https://www.youtube.com/watch?v=BTtyguoPvBU&t=220s) probably 25. Um, this is somewhat

[3:43](https://www.youtube.com/watch?v=BTtyguoPvBU&t=223s) bottlenecked by your ability to talk to

[3:45](https://www.youtube.com/watch?v=BTtyguoPvBU&t=225s) the manager agent and understand what's

[3:47](https://www.youtube.com/watch?v=BTtyguoPvBU&t=227s) happening. Um, uh, I think if you push

[3:51](https://www.youtube.com/watch?v=BTtyguoPvBU&t=231s) too hard on this, you you quickly spin

[3:53](https://www.youtube.com/watch?v=BTtyguoPvBU&t=233s) out of control. Um, I find we can

[3:56](https://www.youtube.com/watch?v=BTtyguoPvBU&t=236s) concurrently manage about 25 builders.

[3:58](https://www.youtube.com/watch?v=BTtyguoPvBU&t=238s) Um, now the key here is that the manager

[4:01](https://www.youtube.com/watch?v=BTtyguoPvBU&t=241s) is talking to this these builders. You

[4:02](https://www.youtube.com/watch?v=BTtyguoPvBU&t=242s) never go into these sessions. You never

[4:04](https://www.youtube.com/watch?v=BTtyguoPvBU&t=244s) talk to them. You never manage them. You

[4:06](https://www.youtube.com/watch?v=BTtyguoPvBU&t=246s) never do anything. Um, it's all done

[4:08](https://www.youtube.com/watch?v=BTtyguoPvBU&t=248s) through the manager agent. And usually

[4:10](https://www.youtube.com/watch?v=BTtyguoPvBU&t=250s) what it's doing is

[4:12](https://www.youtube.com/watch?v=BTtyguoPvBU&t=252s) answering technical questions that don't

[4:14](https://www.youtube.com/watch?v=BTtyguoPvBU&t=254s) I don't need to answer. Um or unblocking

[4:17](https://www.youtube.com/watch?v=BTtyguoPvBU&t=257s) small things. Um I also have an

[4:19](https://www.youtube.com/watch?v=BTtyguoPvBU&t=259s) automation that forces the manager agent

[4:22](https://www.youtube.com/watch?v=BTtyguoPvBU&t=262s) to read all the children sessions every

[4:24](https://www.youtube.com/watch?v=BTtyguoPvBU&t=264s) it's like every 15 minutes. This says

[4:26](https://www.youtube.com/watch?v=BTtyguoPvBU&t=266s) 10, but I've decided 15 is about the

[4:28](https://www.youtube.com/watch?v=BTtyguoPvBU&t=268s) right call. You have to you have to

[4:30](https://www.youtube.com/watch?v=BTtyguoPvBU&t=270s) build an automation. You can't trust

[4:32](https://www.youtube.com/watch?v=BTtyguoPvBU&t=272s) your Androme your manager agent manager

[4:35](https://www.youtube.com/watch?v=BTtyguoPvBU&t=275s) agent to do this right. It's not

[4:38](https://www.youtube.com/watch?v=BTtyguoPvBU&t=278s) deterministic. So this is basically a

[4:40](https://www.youtube.com/watch?v=BTtyguoPvBU&t=280s) script again in Devon. It has

[4:42](https://www.youtube.com/watch?v=BTtyguoPvBU&t=282s) automations. You just tell it, hey,

[4:44](https://www.youtube.com/watch?v=BTtyguoPvBU&t=284s) every 15 minutes go read every single

[4:46](https://www.youtube.com/watch?v=BTtyguoPvBU&t=286s) child, push things along, keep things

[4:49](https://www.youtube.com/watch?v=BTtyguoPvBU&t=289s) moving, right? Um, and then any

[4:51](https://www.youtube.com/watch?v=BTtyguoPvBU&t=291s) questions go back to the manager agent.

[4:53](https://www.youtube.com/watch?v=BTtyguoPvBU&t=293s) So, who's tracking so far? Do you

[4:55](https://www.youtube.com/watch?v=BTtyguoPvBU&t=295s) understand what I'm saying? Okay, good.

[4:57](https://www.youtube.com/watch?v=BTtyguoPvBU&t=297s) And probably a lot of you are doing

[4:59](https://www.youtube.com/watch?v=BTtyguoPvBU&t=299s) this. I'm not claiming this is rocket

[5:00](https://www.youtube.com/watch?v=BTtyguoPvBU&t=300s) science, but this is what we're doing

[5:02](https://www.youtube.com/watch?v=BTtyguoPvBU&t=302s) right now in prod and it's working,

[5:04](https://www.youtube.com/watch?v=BTtyguoPvBU&t=304s) right? Um, each of these builder agents

[5:07](https://www.youtube.com/watch?v=BTtyguoPvBU&t=307s) uh has a a a review agent that it calls

[5:10](https://www.youtube.com/watch?v=BTtyguoPvBU&t=310s) upon. Um, Devon has built-in reviews.

[5:12](https://www.youtube.com/watch?v=BTtyguoPvBU&t=312s) Probably every code harness you have has

[5:14](https://www.youtube.com/watch?v=BTtyguoPvBU&t=314s) built-in reviewers. Um, we typically

[5:17](https://www.youtube.com/watch?v=BTtyguoPvBU&t=317s) stick to a three uh review max where uh

[5:21](https://www.youtube.com/watch?v=BTtyguoPvBU&t=321s) we allow the builders to loop about

[5:23](https://www.youtube.com/watch?v=BTtyguoPvBU&t=323s) three times with reviews. If they still

[5:26](https://www.youtube.com/watch?v=BTtyguoPvBU&t=326s) have any important bugs that have been

[5:27](https://www.youtube.com/watch?v=BTtyguoPvBU&t=327s) fixed, after three, then they bubble

[5:29](https://www.youtube.com/watch?v=BTtyguoPvBU&t=329s) that back up to the manager agent. And

[5:31](https://www.youtube.com/watch?v=BTtyguoPvBU&t=331s) if the manager agent needs to, it

[5:32](https://www.youtube.com/watch?v=BTtyguoPvBU&t=332s) bubbles it back up to me, right? Um,

[5:35](https://www.youtube.com/watch?v=BTtyguoPvBU&t=335s) that seems to work pretty well.

[5:38](https://www.youtube.com/watch?v=BTtyguoPvBU&t=338s) Okay, so what's happening under the code

[5:40](https://www.youtube.com/watch?v=BTtyguoPvBU&t=340s) under under the hood here? Like all the

[5:41](https://www.youtube.com/watch?v=BTtyguoPvBU&t=341s) normal stuff, right? You've got code

[5:42](https://www.youtube.com/watch?v=BTtyguoPvBU&t=342s) plus tests. You've got a QA agent. This

[5:45](https://www.youtube.com/watch?v=BTtyguoPvBU&t=345s) is actually an additional uh process

[5:48](https://www.youtube.com/watch?v=BTtyguoPvBU&t=348s) that we run. Um, it actually spins up a

[5:50](https://www.youtube.com/watch?v=BTtyguoPvBU&t=350s) child of the builder to run QA. Um,

[5:54](https://www.youtube.com/watch?v=BTtyguoPvBU&t=354s) y'all, this is cheap. like you should be

[5:57](https://www.youtube.com/watch?v=BTtyguoPvBU&t=357s) spinning up as many agents as you can as

[5:59](https://www.youtube.com/watch?v=BTtyguoPvBU&t=359s) long as you're not holding up your merge

[6:01](https://www.youtube.com/watch?v=BTtyguoPvBU&t=361s) queue, right? So I this works well for

[6:04](https://www.youtube.com/watch?v=BTtyguoPvBU&t=364s) us now. Um a a wonderful engineer named

[6:07](https://www.youtube.com/watch?v=BTtyguoPvBU&t=367s) Dale recently joined. He helped install

[6:09](https://www.youtube.com/watch?v=BTtyguoPvBU&t=369s) this QA process. I think it's working

[6:11](https://www.youtube.com/watch?v=BTtyguoPvBU&t=371s) pretty well so far. The thing that

[6:12](https://www.youtube.com/watch?v=BTtyguoPvBU&t=372s) you're all going to encounter is just

[6:14](https://www.youtube.com/watch?v=BTtyguoPvBU&t=374s) merge Q slowdown. Um there's this fine

[6:16](https://www.youtube.com/watch?v=BTtyguoPvBU&t=376s) balance of just, you know, pecking your

[6:19](https://www.youtube.com/watch?v=BTtyguoPvBU&t=379s) PRs to death um versus just getting them

[6:22](https://www.youtube.com/watch?v=BTtyguoPvBU&t=382s) merged, right? Um, the other thing I

[6:25](https://www.youtube.com/watch?v=BTtyguoPvBU&t=385s) love, love, love, love is videos. So,

[6:28](https://www.youtube.com/watch?v=BTtyguoPvBU&t=388s) Devon has this built in. Um, if not, you

[6:31](https://www.youtube.com/watch?v=BTtyguoPvBU&t=391s) should build this. Um, it's hard. Um,

[6:34](https://www.youtube.com/watch?v=BTtyguoPvBU&t=394s) every builder records a video of the

[6:36](https://www.youtube.com/watch?v=BTtyguoPvBU&t=396s) feature being used and passing tests.

[6:39](https://www.youtube.com/watch?v=BTtyguoPvBU&t=399s) So, if it's a UI facing feature, you've

[6:42](https://www.youtube.com/watch?v=BTtyguoPvBU&t=402s) got to look at it, right? Um, this is

[6:45](https://www.youtube.com/watch?v=BTtyguoPvBU&t=405s) how we know that what we've built looks

[6:47](https://www.youtube.com/watch?v=BTtyguoPvBU&t=407s) good, how it works, right? So, in Devon,

[6:50](https://www.youtube.com/watch?v=BTtyguoPvBU&t=410s) it's a built-in feature. It's called

[6:51](https://www.youtube.com/watch?v=BTtyguoPvBU&t=411s) test. And basically it it builds test

[6:53](https://www.youtube.com/watch?v=BTtyguoPvBU&t=413s) cases and then it records a video and it

[6:55](https://www.youtube.com/watch?v=BTtyguoPvBU&t=415s) gives you green or or red on those on

[6:57](https://www.youtube.com/watch?v=BTtyguoPvBU&t=417s) those tests. And if you get a video

[6:59](https://www.youtube.com/watch?v=BTtyguoPvBU&t=419s) that's all green, you can watch it and

[7:01](https://www.youtube.com/watch?v=BTtyguoPvBU&t=421s) know what we think is happening is

[7:03](https://www.youtube.com/watch?v=BTtyguoPvBU&t=423s) happening and visually it looks good.

[7:05](https://www.youtube.com/watch?v=BTtyguoPvBU&t=425s) Right? So this is the human in the loop

[7:07](https://www.youtube.com/watch?v=BTtyguoPvBU&t=427s) where a human has to say ship it. So the

[7:11](https://www.youtube.com/watch?v=BTtyguoPvBU&t=431s) manager session will will let me know

[7:13](https://www.youtube.com/watch?v=BTtyguoPvBU&t=433s) there's a video ready for you. Watch it.

[7:15](https://www.youtube.com/watch?v=BTtyguoPvBU&t=435s) Right? So I watched the video and this

[7:17](https://www.youtube.com/watch?v=BTtyguoPvBU&t=437s) is the hard part. you've got to zoom in

[7:19](https://www.youtube.com/watch?v=BTtyguoPvBU&t=439s) and understand like what is this PR

[7:20](https://www.youtube.com/watch?v=BTtyguoPvBU&t=440s) doing? Um what am I watching? Um so I

[7:24](https://www.youtube.com/watch?v=BTtyguoPvBU&t=444s) usually read a summary of like what is

[7:26](https://www.youtube.com/watch?v=BTtyguoPvBU&t=446s) this thing? What is it supposed to do? I

[7:28](https://www.youtube.com/watch?v=BTtyguoPvBU&t=448s) watch it and I ga engage. And I would

[7:30](https://www.youtube.com/watch?v=BTtyguoPvBU&t=450s) encourage you this is the part where you

[7:31](https://www.youtube.com/watch?v=BTtyguoPvBU&t=451s) want to cut corners. You got to watch

[7:33](https://www.youtube.com/watch?v=BTtyguoPvBU&t=453s) the video. Like there is no this is the

[7:36](https://www.youtube.com/watch?v=BTtyguoPvBU&t=456s) part of the bottleneck you have to

[7:38](https://www.youtube.com/watch?v=BTtyguoPvBU&t=458s) engage in as a product owner. Um got to

[7:40](https://www.youtube.com/watch?v=BTtyguoPvBU&t=460s) watch the video, got to pay attention.

[7:42](https://www.youtube.com/watch?v=BTtyguoPvBU&t=462s) Usually it has to be on your laptop. Um

[7:45](https://www.youtube.com/watch?v=BTtyguoPvBU&t=465s) I I I this literally before this talk I

[7:47](https://www.youtube.com/watch?v=BTtyguoPvBU&t=467s) was on my phone you know talking to my

[7:49](https://www.youtube.com/watch?v=BTtyguoPvBU&t=469s) manager my manager agent and it said

[7:51](https://www.youtube.com/watch?v=BTtyguoPvBU&t=471s) there's a video for you. I watch it and

[7:53](https://www.youtube.com/watch?v=BTtyguoPvBU&t=473s) I just couldn't see it well enough and I

[7:55](https://www.youtube.com/watch?v=BTtyguoPvBU&t=475s) was so [snorts] tempted to be like it's

[7:56](https://www.youtube.com/watch?v=BTtyguoPvBU&t=476s) okay ship [laughter] it like no get back

[8:00](https://www.youtube.com/watch?v=BTtyguoPvBU&t=480s) to my machine watch the watch the video.

[8:02](https://www.youtube.com/watch?v=BTtyguoPvBU&t=482s) Um this is working like you know we we

[8:06](https://www.youtube.com/watch?v=BTtyguoPvBU&t=486s) ship a lot of PRs uh at high quality

[8:09](https://www.youtube.com/watch?v=BTtyguoPvBU&t=489s) this way. We've iterated, you know,

[8:11](https://www.youtube.com/watch?v=BTtyguoPvBU&t=491s) hundreds of times on this process and I

[8:13](https://www.youtube.com/watch?v=BTtyguoPvBU&t=493s) I feel pretty good about this. Now, this

[8:14](https://www.youtube.com/watch?v=BTtyguoPvBU&t=494s) will change tomorrow [laughter] and

[8:17](https://www.youtube.com/watch?v=BTtyguoPvBU&t=497s) probably will be useless a week from

[8:18](https://www.youtube.com/watch?v=BTtyguoPvBU&t=498s) now. Uh, but it works well today. Um,

[8:21](https://www.youtube.com/watch?v=BTtyguoPvBU&t=501s) uh, the thing I will say is like if

[8:23](https://www.youtube.com/watch?v=BTtyguoPvBU&t=503s) you're coding locally, stop. Like, you

[8:25](https://www.youtube.com/watch?v=BTtyguoPvBU&t=505s) really need to be coding with a a cloud

[8:27](https://www.youtube.com/watch?v=BTtyguoPvBU&t=507s) agent. I don't care if it's Devon or

[8:29](https://www.youtube.com/watch?v=BTtyguoPvBU&t=509s) whatever it is, but if you're running

[8:31](https://www.youtube.com/watch?v=BTtyguoPvBU&t=511s) local dev, you are screwing up. You're

[8:34](https://www.youtube.com/watch?v=BTtyguoPvBU&t=514s) holding yourself back. You're holding

[8:35](https://www.youtube.com/watch?v=BTtyguoPvBU&t=515s) your team back. You're holding

[8:36](https://www.youtube.com/watch?v=BTtyguoPvBU&t=516s) everything back because you can't do

[8:38](https://www.youtube.com/watch?v=BTtyguoPvBU&t=518s) this. Um so that's where we're going and

[8:41](https://www.youtube.com/watch?v=BTtyguoPvBU&t=521s) I feel strongly about that.

[8:43](https://www.youtube.com/watch?v=BTtyguoPvBU&t=523s) So how does this work at scale? Well,

[8:45](https://www.youtube.com/watch?v=BTtyguoPvBU&t=525s) you know, say you have 10 folks um uh in

[8:49](https://www.youtube.com/watch?v=BTtyguoPvBU&t=529s) the who are working on things. Typically

[8:51](https://www.youtube.com/watch?v=BTtyguoPvBU&t=531s) they'll have an engineering manager uh

[8:54](https://www.youtube.com/watch?v=BTtyguoPvBU&t=534s) session each. Sometimes they'll have two

[8:56](https://www.youtube.com/watch?v=BTtyguoPvBU&t=536s) like right now I have two running. Um

[8:59](https://www.youtube.com/watch?v=BTtyguoPvBU&t=539s) and what's beautiful is that these

[9:01](https://www.youtube.com/watch?v=BTtyguoPvBU&t=541s) things talk to each other. And so uh

[9:04](https://www.youtube.com/watch?v=BTtyguoPvBU&t=544s) again in Devon this is built in. It's

[9:06](https://www.youtube.com/watch?v=BTtyguoPvBU&t=546s) probably built into whatever harness you

[9:08](https://www.youtube.com/watch?v=BTtyguoPvBU&t=548s) use. the sessions all need to be able to

[9:10](https://www.youtube.com/watch?v=BTtyguoPvBU&t=550s) talk to each other and they all need to

[9:11](https://www.youtube.com/watch?v=BTtyguoPvBU&t=551s) know who each other is, right? And

[9:14](https://www.youtube.com/watch?v=BTtyguoPvBU&t=554s) they're really good at this. Like this

[9:16](https://www.youtube.com/watch?v=BTtyguoPvBU&t=556s) is such a delight. So what you'll see if

[9:19](https://www.youtube.com/watch?v=BTtyguoPvBU&t=559s) if you if you zoom into any sessions,

[9:21](https://www.youtube.com/watch?v=BTtyguoPvBU&t=561s) they're messaging each other. It's

[9:23](https://www.youtube.com/watch?v=BTtyguoPvBU&t=563s) clear, it's straightforward, it's fast,

[9:26](https://www.youtube.com/watch?v=BTtyguoPvBU&t=566s) it's good. Like you should be allowing

[9:29](https://www.youtube.com/watch?v=BTtyguoPvBU&t=569s) your sessions to talk to each other. Um

[9:31](https://www.youtube.com/watch?v=BTtyguoPvBU&t=571s) and again, if you're not, you are

[9:33](https://www.youtube.com/watch?v=BTtyguoPvBU&t=573s) holding yourself back and you're

[9:34](https://www.youtube.com/watch?v=BTtyguoPvBU&t=574s) probably shipping worse software. Um so

[9:36](https://www.youtube.com/watch?v=BTtyguoPvBU&t=576s) they all talk a lot. Um, and it's not

[9:38](https://www.youtube.com/watch?v=BTtyguoPvBU&t=578s) chitchat or stupid stuff like this is

[9:41](https://www.youtube.com/watch?v=BTtyguoPvBU&t=581s) important stuff that they need to know

[9:42](https://www.youtube.com/watch?v=BTtyguoPvBU&t=582s) to unblock each other. Um, which is very

[9:45](https://www.youtube.com/watch?v=BTtyguoPvBU&t=585s) useful. However, you still need what we

[9:48](https://www.youtube.com/watch?v=BTtyguoPvBU&t=588s) call a merge Q manager. Um,

[9:51](https://www.youtube.com/watch?v=BTtyguoPvBU&t=591s) so this is how we coordinate this just

[9:55](https://www.youtube.com/watch?v=BTtyguoPvBU&t=595s) wave of PRs that come in. You know, you

[9:59](https://www.youtube.com/watch?v=BTtyguoPvBU&t=599s) have things like database migration

[10:01](https://www.youtube.com/watch?v=BTtyguoPvBU&t=601s) numbers. These need to be coordinated.

[10:03](https://www.youtube.com/watch?v=BTtyguoPvBU&t=603s) Like if all the agents are trying to

[10:04](https://www.youtube.com/watch?v=BTtyguoPvBU&t=604s) grab these things, you constantly

[10:06](https://www.youtube.com/watch?v=BTtyguoPvBU&t=606s) collide, right? So the merge manager

[10:08](https://www.youtube.com/watch?v=BTtyguoPvBU&t=608s) just helps all that go away. It also is

[10:11](https://www.youtube.com/watch?v=BTtyguoPvBU&t=611s) a little more intelligent about how the

[10:13](https://www.youtube.com/watch?v=BTtyguoPvBU&t=613s) merge Q is happening. So this is really

[10:15](https://www.youtube.com/watch?v=BTtyguoPvBU&t=615s) nice. Um it's working well for us. And

[10:18](https://www.youtube.com/watch?v=BTtyguoPvBU&t=618s) again, these are just sessions. This is

[10:20](https://www.youtube.com/watch?v=BTtyguoPvBU&t=620s) not like something special. These are

[10:22](https://www.youtube.com/watch?v=BTtyguoPvBU&t=622s) just sessions in Devon and I and I name

[10:25](https://www.youtube.com/watch?v=BTtyguoPvBU&t=625s) them so I understand what's happening.

[10:27](https://www.youtube.com/watch?v=BTtyguoPvBU&t=627s) Um and it's pretty straightforward. You

[10:30](https://www.youtube.com/watch?v=BTtyguoPvBU&t=630s) know, the talk from the gentleman from

[10:31](https://www.youtube.com/watch?v=BTtyguoPvBU&t=631s) OpenAI, you know, said basically you got

[10:33](https://www.youtube.com/watch?v=BTtyguoPvBU&t=633s) to unhobble. Just unhobble the thing,

[10:35](https://www.youtube.com/watch?v=BTtyguoPvBU&t=635s) right? So set up the roles and then

[10:39](https://www.youtube.com/watch?v=BTtyguoPvBU&t=639s) unhobble the model, right? Um, and I

[10:41](https://www.youtube.com/watch?v=BTtyguoPvBU&t=641s) think just having a merge manager say,

[10:43](https://www.youtube.com/watch?v=BTtyguoPvBU&t=643s) "Hey, make all this better." Uh, really

[10:45](https://www.youtube.com/watch?v=BTtyguoPvBU&t=645s) works well. All right. Who's tracking so

[10:47](https://www.youtube.com/watch?v=BTtyguoPvBU&t=647s) far? Good. Who thinks this is [&nbsp;__&nbsp;]

[10:52](https://www.youtube.com/watch?v=BTtyguoPvBU&t=652s) Raise your hand. Oh, one guy. Two. Wait.

[10:54](https://www.youtube.com/watch?v=BTtyguoPvBU&t=654s) One. Come on. Be be more brave. We have

[10:58](https://www.youtube.com/watch?v=BTtyguoPvBU&t=658s) to talk afterwards. Where Where are you?

[11:00](https://www.youtube.com/watch?v=BTtyguoPvBU&t=660s) Um, okay. And then we use GitHub merge

[11:02](https://www.youtube.com/watch?v=BTtyguoPvBU&t=662s) cues. It's fine. It's not rocket

[11:04](https://www.youtube.com/watch?v=BTtyguoPvBU&t=664s) science. It works pretty well. So,

[11:07](https://www.youtube.com/watch?v=BTtyguoPvBU&t=667s) all right. Next up, how we hire. Um, so

[11:10](https://www.youtube.com/watch?v=BTtyguoPvBU&t=670s) who's hiring engineers or employees

[11:12](https://www.youtube.com/watch?v=BTtyguoPvBU&t=672s) right now? Raise your hand. Yeah, decent

[11:15](https://www.youtube.com/watch?v=BTtyguoPvBU&t=675s) number of you. Um, I think everything

[11:17](https://www.youtube.com/watch?v=BTtyguoPvBU&t=677s) has changed. Every everything you used

[11:19](https://www.youtube.com/watch?v=BTtyguoPvBU&t=679s) to do about hiring employees is over and

[11:22](https://www.youtube.com/watch?v=BTtyguoPvBU&t=682s) none of it makes sense. Um, it's all

[11:24](https://www.youtube.com/watch?v=BTtyguoPvBU&t=684s) outdated and I would encourage you to do

[11:26](https://www.youtube.com/watch?v=BTtyguoPvBU&t=686s) something like this. So, we don't do

[11:30](https://www.youtube.com/watch?v=BTtyguoPvBU&t=690s) number one any sort of pre-screens. We

[11:32](https://www.youtube.com/watch?v=BTtyguoPvBU&t=692s) don't do any video chats. We don't do

[11:34](https://www.youtube.com/watch?v=BTtyguoPvBU&t=694s) anything with people um it for most of

[11:37](https://www.youtube.com/watch?v=BTtyguoPvBU&t=697s) this process because it will bias you.

[11:40](https://www.youtube.com/watch?v=BTtyguoPvBU&t=700s) You'll like their hobbies. You'll like

[11:42](https://www.youtube.com/watch?v=BTtyguoPvBU&t=702s) the way they talk to you. You'll think

[11:44](https://www.youtube.com/watch?v=BTtyguoPvBU&t=704s) they're interesting. And it's all noise.

[11:46](https://www.youtube.com/watch?v=BTtyguoPvBU&t=706s) None of it is signal, right? All that

[11:48](https://www.youtube.com/watch?v=BTtyguoPvBU&t=708s) matters is can they operate an agent,

[11:51](https://www.youtube.com/watch?v=BTtyguoPvBU&t=711s) right? Can they do that? So, the way we

[11:54](https://www.youtube.com/watch?v=BTtyguoPvBU&t=714s) do that is we ask candidates to submit a

[11:57](https://www.youtube.com/watch?v=BTtyguoPvBU&t=717s) 45ish minute video, whole screen

[12:00](https://www.youtube.com/watch?v=BTtyguoPvBU&t=720s) recording. Um, and we ask them to ship a

[12:03](https://www.youtube.com/watch?v=BTtyguoPvBU&t=723s) feature PR typically on a app that they

[12:06](https://www.youtube.com/watch?v=BTtyguoPvBU&t=726s) already maintain and we just we just ask

[12:09](https://www.youtube.com/watch?v=BTtyguoPvBU&t=729s) them to record that. No instruction, no

[12:12](https://www.youtube.com/watch?v=BTtyguoPvBU&t=732s) guidance, no nothing. And we just want

[12:14](https://www.youtube.com/watch?v=BTtyguoPvBU&t=734s) to see like how good of an agent

[12:16](https://www.youtube.com/watch?v=BTtyguoPvBU&t=736s) operator is this person, right? So, um,

[12:20](https://www.youtube.com/watch?v=BTtyguoPvBU&t=740s) that is free. We don't pay for that.

[12:21](https://www.youtube.com/watch?v=BTtyguoPvBU&t=741s) That's basically how you get in the

[12:23](https://www.youtube.com/watch?v=BTtyguoPvBU&t=743s) door. Um, and what we then do is we feed

[12:26](https://www.youtube.com/watch?v=BTtyguoPvBU&t=746s) these videos into a ranking agent and

[12:28](https://www.youtube.com/watch?v=BTtyguoPvBU&t=748s) basically say watch the video, watch the

[12:30](https://www.youtube.com/watch?v=BTtyguoPvBU&t=750s) transcript, stack rank these people. Um,

[12:33](https://www.youtube.com/watch?v=BTtyguoPvBU&t=753s) now this what's interesting is I did

[12:35](https://www.youtube.com/watch?v=BTtyguoPvBU&t=755s) this and I didn't trust it the first

[12:37](https://www.youtube.com/watch?v=BTtyguoPvBU&t=757s) time. I didn't like what it said. Um,

[12:41](https://www.youtube.com/watch?v=BTtyguoPvBU&t=761s) and what's ironic is it actually was

[12:43](https://www.youtube.com/watch?v=BTtyguoPvBU&t=763s) correct. Like the person that was number

[12:45](https://www.youtube.com/watch?v=BTtyguoPvBU&t=765s) one, I I should have hired them. Um, uh,

[12:48](https://www.youtube.com/watch?v=BTtyguoPvBU&t=768s) but I didn't trust it because I was I

[12:50](https://www.youtube.com/watch?v=BTtyguoPvBU&t=770s) was biased. So you stack rank them. Now,

[12:53](https://www.youtube.com/watch?v=BTtyguoPvBU&t=773s) this is just a lot of video to watch.

[12:54](https://www.youtube.com/watch?v=BTtyguoPvBU&t=774s) You could watch all these videos if you

[12:56](https://www.youtube.com/watch?v=BTtyguoPvBU&t=776s) want. If you have time, do it. But I

[12:58](https://www.youtube.com/watch?v=BTtyguoPvBU&t=778s) think you'll find actually a very

[13:00](https://www.youtube.com/watch?v=BTtyguoPvBU&t=780s) intelligent model is pretty good at

[13:01](https://www.youtube.com/watch?v=BTtyguoPvBU&t=781s) understanding like how how good is this

[13:03](https://www.youtube.com/watch?v=BTtyguoPvBU&t=783s) person. Um, what's interesting though is

[13:06](https://www.youtube.com/watch?v=BTtyguoPvBU&t=786s) uh candidates will mumble um or uh

[13:10](https://www.youtube.com/watch?v=BTtyguoPvBU&t=790s) sometimes or they won't think out loud,

[13:11](https://www.youtube.com/watch?v=BTtyguoPvBU&t=791s) which is kind of fascinating. So, you

[13:13](https://www.youtube.com/watch?v=BTtyguoPvBU&t=793s) can't really tell what they're thinking

[13:14](https://www.youtube.com/watch?v=BTtyguoPvBU&t=794s) or doing. Um, which is interesting. So,

[13:16](https://www.youtube.com/watch?v=BTtyguoPvBU&t=796s) that's uh something to sort of note. Um,

[13:19](https://www.youtube.com/watch?v=BTtyguoPvBU&t=799s) and some talk a lot. [laughter] So, you

[13:23](https://www.youtube.com/watch?v=BTtyguoPvBU&t=803s) never know. Uh, then we basically pick

[13:25](https://www.youtube.com/watch?v=BTtyguoPvBU&t=805s) the top five and we invite them to a

[13:27](https://www.youtube.com/watch?v=BTtyguoPvBU&t=807s) paid project. So, we pay them a fixed

[13:28](https://www.youtube.com/watch?v=BTtyguoPvBU&t=808s) fee for 16 hours of their time. We

[13:31](https://www.youtube.com/watch?v=BTtyguoPvBU&t=811s) actually give them a seat on Devon and

[13:33](https://www.youtube.com/watch?v=BTtyguoPvBU&t=813s) we ask them to uh to get a real PR ready

[13:37](https://www.youtube.com/watch?v=BTtyguoPvBU&t=817s) for merge. Um, we have them sign an NDA.

[13:40](https://www.youtube.com/watch?v=BTtyguoPvBU&t=820s) Um, and we record all the video. This is

[13:44](https://www.youtube.com/watch?v=BTtyguoPvBU&t=824s) 16 hours of video. Um, we basically use

[13:46](https://www.youtube.com/watch?v=BTtyguoPvBU&t=826s) Loom and then uh we feed all the videos

[13:49](https://www.youtube.com/watch?v=BTtyguoPvBU&t=829s) into uh our our agent basically Devon

[13:52](https://www.youtube.com/watch?v=BTtyguoPvBU&t=832s) and we say hey who you know which one

[13:54](https://www.youtube.com/watch?v=BTtyguoPvBU&t=834s) should we pick? I actually watched all

[13:57](https://www.youtube.com/watch?v=BTtyguoPvBU&t=837s) 16 hours of these videos for the last

[13:59](https://www.youtube.com/watch?v=BTtyguoPvBU&t=839s) round. Uh it was brutal but it did give

[14:01](https://www.youtube.com/watch?v=BTtyguoPvBU&t=841s) me a good read on the people. Um and

[14:04](https://www.youtube.com/watch?v=BTtyguoPvBU&t=844s) then you basically pick one. You pick

[14:06](https://www.youtube.com/watch?v=BTtyguoPvBU&t=846s) your finalists and then what's important

[14:09](https://www.youtube.com/watch?v=BTtyguoPvBU&t=849s) is that you don't do calls before this.

[14:12](https://www.youtube.com/watch?v=BTtyguoPvBU&t=852s) Um you will get biased. you'll get

[14:14](https://www.youtube.com/watch?v=BTtyguoPvBU&t=854s) confused. You'll try to pick somebody

[14:16](https://www.youtube.com/watch?v=BTtyguoPvBU&t=856s) you shouldn't. And I would just call the

[14:19](https://www.youtube.com/watch?v=BTtyguoPvBU&t=859s) finalists and say, "Hey, you know, we

[14:22](https://www.youtube.com/watch?v=BTtyguoPvBU&t=862s) think we'd like to hire you. Let's talk

[14:23](https://www.youtube.com/watch?v=BTtyguoPvBU&t=863s) a little bit." Um, and that gives you

[14:26](https://www.youtube.com/watch?v=BTtyguoPvBU&t=866s) the final sort of human read, but you're

[14:27](https://www.youtube.com/watch?v=BTtyguoPvBU&t=867s) going to watch videos of these people

[14:28](https://www.youtube.com/watch?v=BTtyguoPvBU&t=868s) for hours. Like, you're not really going

[14:30](https://www.youtube.com/watch?v=BTtyguoPvBU&t=870s) to need that call. Um, but I think

[14:32](https://www.youtube.com/watch?v=BTtyguoPvBU&t=872s) there's a little bit of of of human

[14:33](https://www.youtube.com/watch?v=BTtyguoPvBU&t=873s) connection there that's important.

[14:36](https://www.youtube.com/watch?v=BTtyguoPvBU&t=876s) I think this is how everyone's going to

[14:37](https://www.youtube.com/watch?v=BTtyguoPvBU&t=877s) hire. Like, this works. Um, you know,

[14:40](https://www.youtube.com/watch?v=BTtyguoPvBU&t=880s) I've hired hundreds and hundreds and

[14:42](https://www.youtube.com/watch?v=BTtyguoPvBU&t=882s) hundreds of people in my career.

[14:44](https://www.youtube.com/watch?v=BTtyguoPvBU&t=884s) This is just way easier and way better.

[14:46](https://www.youtube.com/watch?v=BTtyguoPvBU&t=886s) Um, and I think it's where we're all

[14:48](https://www.youtube.com/watch?v=BTtyguoPvBU&t=888s) moving to. So,

[14:52](https://www.youtube.com/watch?v=BTtyguoPvBU&t=892s) I'm actually going to finish 3 minutes

[14:53](https://www.youtube.com/watch?v=BTtyguoPvBU&t=893s) early. Um, if you want to know more

[14:56](https://www.youtube.com/watch?v=BTtyguoPvBU&t=896s) about Untangle, just go to Untangle. Um,

[14:58](https://www.youtube.com/watch?v=BTtyguoPvBU&t=898s) I'm on X like just all the time. I

[15:00](https://www.youtube.com/watch?v=BTtyguoPvBU&t=900s) literally been on X for 20 years. Um,

[15:03](https://www.youtube.com/watch?v=BTtyguoPvBU&t=903s) and so, uh, that's where I live and

[15:05](https://www.youtube.com/watch?v=BTtyguoPvBU&t=905s) breathe. So, if you want to know more

[15:06](https://www.youtube.com/watch?v=BTtyguoPvBU&t=906s) about uh leading companies, managing

[15:09](https://www.youtube.com/watch?v=BTtyguoPvBU&t=909s) agents, um that's where I am. If you're

[15:12](https://www.youtube.com/watch?v=BTtyguoPvBU&t=912s) interested in family law and you're

[15:14](https://www.youtube.com/watch?v=BTtyguoPvBU&t=914s) passionate about making divorce less

[15:16](https://www.youtube.com/watch?v=BTtyguoPvBU&t=916s) painful, um get in touch. Um and that is

[15:19](https://www.youtube.com/watch?v=BTtyguoPvBU&t=919s) all I have to say. Thanks everybody.

[15:21](https://www.youtube.com/watch?v=BTtyguoPvBU&t=921s) Appreciate it. [applause]
