# Rethinking Claude Code Best Practices — Lydia Hallie, Anthropic

Lydia Hallie, Anthropic

[Talk page](https://ai.engineer/talks/vlfCjWuv4es-rethinking-claude-code-best-practices) · [Video](https://www.youtube.com/watch?v=vlfCjWuv4es)

## Rethinking Claude Code Best Practices

Lydia Hallie follows a Claude Code session from manual approvals to hours of unattended work. A staging push, a test runner and a login migration show where a stronger model helps and which decisions still belong in permissions, the operating system or a test.

### From correcting one session to managing dozens

[0:00](https://www.youtube.com/watch?v=vlfCjWuv4es&t=0s)

Earlier in the year, a Claude Code session often stopped before the job was done. The developer was there to nudge it along, correct a decision and add another emphatic reminder to `CLAUDE.md`. Before auto mode, the choice was to approve commands by hand or accept the risk of skipping permissions. Lydia Hallie, who works on Claude Code at Anthropic, remembers doing this for one or two sessions at a time.

By the time of the talk, developers at Anthropic were running dozens of sessions concurrently. A session could work for hours, check its own results and even coordinate other sessions. Hallie puts the volume in perspective with a company-wide count of about 2,500 changes a day in May and more than 30,000 a day in September. At that scale, who catches a mistake while the developer is looking at another session?

Hallie rebuilds one of the earlier sessions by changing two defaults. She moves it from **Opus 4.5** to **Opus 5.5**, which needs fewer mid-session corrections because it can infer more from the request and the codebase. She also switches manual permissions to auto mode, which Anthropic began making the default in August. The model takes on more of the steering, and auto mode handles routine approvals. Two questions remain. What can an approved action affect? How will anyone know whether the finished code does the right thing?

### Keep the project knowledge Claude cannot discover

[3:14](https://www.youtube.com/watch?v=vlfCjWuv4es&t=194s)

An instruction that helped an older model can become a distraction. Hallie describes newer models following emphatic reminders off track, making extra tool calls and producing a worse result. Her editing question for `CLAUDE.md` is whether the instruction tells Claude something it could reasonably miss by reading the project.

Suppose a service still depends on a legacy data source that has no direct reference in the code. An engineer inspecting the repository could miss that dependency. So could Claude. Keep it in `CLAUDE.md`. A description of the tech stack or directions to an obvious part of the repository can usually go. Claude Code’s prompt audit checks both `CLAUDE.md` and the project’s skills for instructions the selected model no longer needs.

![A five-line CLAUDE.md excerpt highlights its final line in green. It says that src/legacy still serves the mobile apps.](https://ai.engineer/api/talk-essay-media/06764c7e260fd10d964d3cdfdf759a5bc9a83e9d65ea3fc9c20c502d3d178097)

The highlighted instruction preserves a constraint the repository cannot explain on its own: an older mobile app still needs the legacy code.

### Why a staging push can stop in auto mode

[4:12](https://www.youtube.com/watch?v=vlfCjWuv4es&t=252s)

Auto mode uses a separate classifier for most actions beyond reading and editing files in the project. The coding model proposes an action. The classifier can allow a familiar operation, such as running tests or installing dependencies, and block a command whose safety is uncertain.

Consider a request to ship a change to staging. The developer knows that deployment entails pushing to a separate deploy repository and would ordinarily approve the push. Auto mode starts with trust in the current folder and repository. The deploy repository falls outside that scope, so the push stops and the developer is back at the keyboard.

Where can the team record what it knows about that destination? Auto mode’s settings accept plain English naming trusted repositories, buckets and domains, along with actions to allow or reject. Those settings go to the classifier. Naming the deploy repository gives it the missing fact about this push while keeping the scope of that trust explicit.

### Use reversibility to decide what needs a deny rule

[5:32](https://www.youtube.com/watch?v=vlfCjWuv4es&t=332s)

In an August study Hallie cites, about 14% of people caught a clearly dangerous command. Auto mode blocked 89% of the dangerous commands in the study. That still left roughly one in ten getting through. A team that needs to prohibit a particular tool call can put a deny rule in the Claude Code harness, the software that receives tool calls and decides whether they run. The harness checks the rule first. If the call matches, it stops there, before the classifier ever sees it.

Hallie’s test is whether you could take the action back if Claude got it wrong once. You can revert a bad code change. Discarding uncommitted work may destroy the only copy. Reading a secret key can put its value into the session transcript. Those last two cases justify a rule ahead of time.

Take the project’s local environment file. Auto mode would ordinarily permit a read. Adding a deny rule for that file lets the harness stop a matching request, including a direct `cat` of the file. That handles the case where the proposed command identifies the file it wants to open.

### What can an approved test process read?

[7:16](https://www.youtube.com/watch?v=vlfCjWuv4es&t=436s)

`npm test` is the sort of command a developer and auto mode would both approve. Once it starts, it executes project code and dependencies. A compromised dependency could try to read the same environment file or the developer’s SSH keys and send their contents to an unfamiliar host. All the developer saw at approval time was `npm test`. The dependency’s file access happens inside the running process.

Where could that second read be stopped while still letting Claude test its work? With the local shell sandbox enabled, the file restriction also goes to the operating system. The test can start. When its dependency tries to open the protected file, the operating system refuses the read. Child processes inherit the restrictions, so the same limit applies if the test launches another program.

Sandbox settings govern which files a command can read, which directories it can write to and which network hosts it can reach. Most of the local machine is still readable by default in Hallie’s setup, so protecting SSH keys takes its own rule. The scope also matters. The sandbox covers shell commands and their descendants. Claude’s own file operations, MCP servers and hooks still run outside it on the developer’s machine.

Where does each attempt to read the same file stop?
A direct read names the file before execution. In the test path, the filename becomes relevant only after the dependency runs, when the operating system can enforce the restriction.
- Direct file read — The proposed command names the protected environment file.
- Harness blocks the call — A matching deny rule ends the request before classification.
- Run npm test — The proposed command names the test runner.
- Auto mode allows the test — The test process can start.
- Dependency requests the file — The attempted read occurs inside the running test.
- Operating system refuses the read — The sandbox applies the file restriction to the process.
- Direct file read → Harness blocks the call (Matches the deny rule)
- Run npm test → Auto mode allows the test (Recognized test command)
- Auto mode allows the test → Dependency requests the file (Test executes dependency code)
- Dependency requests the file → Operating system refuses the read (Attempts to open the protected file)

### Give the whole session a fresh machine

[9:26](https://www.youtube.com/watch?v=vlfCjWuv4es&t=566s)

A cloud session gets its own machine with a fresh copy of the repository. It keeps running when the developer closes the laptop. In Hallie’s example, the local environment file, SSH keys and GitHub token were never in Git, so none arrive with the checkout. Even a tool outside the shell sandbox would find no local copy of those secrets there.

If the session discovers it needs an API key, it can flag the missing credential. The developer can supply a network secret so the session can use the key without sending its value to the model. Network access has a separate setting. The defaults Hallie describes include package registries and GitHub. Reaching another host requires adding its domain to the environment.

Disposing of the machine leaves a pushed commit in its remote repository and a message sent through a connector with its recipient. Hallie keeps auto mode enabled and commits the project’s deny rules to the repository. Each fresh checkout brings those rules along, so checks on outgoing actions can still catch a force push or a secret in a commit.

![A cloud outline encloses a timeline marked with the npm test command. Two captions explain that the run uses a fresh repository copy and can continue with the laptop closed.](https://ai.engineer/api/talk-essay-media/6947ce18d7cfbec8984cc862fdb3059b0cb510644c0dc14adc8165e6eaa84982)

The cloud session runs tests on its own copy of the repository and can continue when the developer closes their laptop.

### Does the API migration keep existing users signed in?

[10:47](https://www.youtube.com/watch?v=vlfCjWuv4es&t=647s)

An isolated session can still spend hours implementing the wrong behavior. Consider a request to move login to a new API. An engineer watching each change could catch an unwanted decision as it happened. An engineer coming back hours later needs a way to tell what the implementation did to the people using the old API.

What would count as migrated? Hallie gives three plausible outcomes from the same prompt. One run could move login over while leaving the old API in place. Another could switch only new logins. A third could complete the switch and log everyone out. A successful login by a new user would tell a reviewer very little about someone who was already signed in.

If existing sessions must survive, make that behavior the first pass-or-fail check. The implementation that signs everyone out fails there, sending Claude back to find another way. Other requirements may be observable through a build or a screenshot. Reviewing a few well-chosen checks takes less work than reconstructing each decision from the final diff.

The simplest place to specify the check is in the request. Describe what done looks like, ask Claude to break the work into tasks with their own tests, and ask it to show the actual output. That last request gives the returning reviewer something to inspect.

### Make a failed check block completion

[12:21](https://www.youtube.com/watch?v=vlfCjWuv4es&t=741s)

A prompt leaves it up to the model to run a check. For checks that must run, Claude Code provides hooks. A hook is a script the harness invokes at a specified event. The team chooses when it runs, which command it executes and what it checks.

- **After a file edit.** Run the linter so Claude gets feedback on the edit it just made.
- **When Claude tries to complete a task.** Run the tests for the code it touched. A change to authentication has to pass the authentication tests.
- **When Claude tries to finish the session.** Run the whole test suite so a failure elsewhere can send Claude back to fix the change.

What turns a failing test into an unfinished task? Many test tools report failure with exit code `1`. In the task-completion hook Hallie describes, passing `1` through still lets Claude Code mark the task complete. The hook script needs to translate the test failure into exit code `2`. That tells the harness to block completion and sends Claude back to fix the problem.

### Give every pull request the same checks

[13:25](https://www.youtube.com/watch?v=vlfCjWuv4es&t=805s)

Once one session can work unattended, a developer can start several. Each cloud session has its own machine and checkout, which saves the developer from managing local worktrees to keep concurrent edits apart. Starting each session personally and reviewing each result is a reasonable stopping point. The developer still chooses which work to start and when to accept it.

The pull request gives those independent sessions a common checkpoint. Continuous integration (CI) runs the same checks on every PR, regardless of how the originating session was configured or which tests Claude ran along the way. With CI autofix enabled, Claude can see a failed check and go back to fix the change. The reviewer can spend that time on decisions the checks leave open.

For selected changes, GitHub’s merge-when-ready option can merge the PR once its checks pass. Hallie’s team bases that choice on risk. Every PR first gets a Claude review from a session with fresh context. A separate group of agents double-checks each bug it reports. A small documentation or copy edit can proceed on green automated checks alone. An ordinary product change may need one human reviewer, and a risky change may need two.

### Let one session coordinate the others

[15:10](https://www.youtube.com/watch?v=vlfCjWuv4es&t=910s)

Who keeps track when eight sessions come back with results? Hallie finds that much context switching difficult. In Claude Code Projects, in beta at the time of the talk, the developer gives one main session the job. That session acts as an orchestrator. It splits the job into tasks, starts a separate session or *thread* for each task, tracks what comes back and verifies the results. The developer has one conversation to follow.

Projects lets the developer choose the models for those two roles independently. Hallie gives **Fable** as an example for the main session, where the job is deciding what to do and judging the work that comes back. An incomplete result goes back to its thread for revision until it passes. Opus 5.5 is her example for the threads that write the code. Projects also manages context automatically, taking over decisions about when to compact or clear a session.

Hallie’s suggested first experiment is one task considerably bigger than you usually give Claude. Check that CI catches the failures you care about. Configure the cloud environment’s network access, secrets and setup scripts, then start the session and see where it needs you again. In the staging example, the push waits for a decision about which repository to trust. In the login example, the session-preservation check fails and sends Claude back to revise the migration. One calls for project knowledge from the developer. The other is a correction the session can make while the developer is away.

## Key takeaways

- Keep the unreferenced legacy dependency in `CLAUDE.md`. Revisit directions the model can infer from the repository when auditing instructions and skills after an upgrade.
- Match the control to when the risk occurs. A deny rule stops a matching direct file read before execution. A sandbox can stop a dependency from opening that file inside an approved test run.
- Choose a check that distinguishes the intended behavior from a plausible wrong result. For the login migration, exercise an existing session. To make a failing test block task completion, have the hook return exit code `2`.

## Slides

### Rethinking Claude Code Best Practices

Lydia Hallie of Anthropic’s Claude Code team opens with a diagram. One orange line connects five person icons, then branches into paths marked with green checks.

[0:00](https://www.youtube.com/watch?v=vlfCjWuv4es&t=0s) [Title slide crediting Lydia Hallie. An orange line connects five person icons before branching into paths marked with green checks.](https://assets.ai.engineer/sites/aiecode2025/sha256/a0e71a70140a32e7df6eef09d7348b075aa0eed9d2529aaead5f0a3716ebb2e5.webp)

Rethinking
Claude Code
Best Practices

Lydia Hallie

### Manual steering and command approval in Claude Code

While Claude works on a failing auth test, the developer steers it toward a code fix and repeats the auth helper instruction in CLAUDE.md. Claude then asks permission to run git reset --hard, which would discard uncommitted changes.

[1:01](https://www.youtube.com/watch?v=vlfCjWuv4es&t=61s) [A failing-test timeline shows corrections above the edits and test run. A dialog asks whether to discard uncommitted changes, with Deny, Always allow and Allow once options.](https://assets.ai.engineer/sites/aiecode2025/sha256/37690135ecaf2a1981ad0d51a30b9ab15877e3fad5aa48513ab2faa28d59bf34.webp)

auth.test.ts is failing, can you fix it?

Fix the code, not the test
Use our auth helper

CLAUDE.md
IMPORTANT: always use the auth helper

Read
Edit ✓
Edit ✓
Bash npm test ✓
Edit ✓
Edit ✓
Bash git reset --hard

Allow Claude to run this command?
Discard uncommitted changes
$ git reset --hard

Deny
Always allow
Allow once

### Eight parallel Claude Code sessions

In this example, Claude splits one weekly task list into eight parallel sessions. Each checks its own work. Over 6h 30m, pull requests move through review or auto-merge, and the task list shows which changes have merged.

[1:23](https://www.youtube.com/watch?v=vlfCjWuv4es&t=83s) [A weekly task list branches into eight cloud sessions under a 6 h 30 m label. The diagram traces pull requests through review or auto-merge, and the task list shows four merged items.](https://assets.ai.engineer/sites/aiecode2025/sha256/1174bc761ffc6535a72e00fbebc3c0ed41e457d88b62a3e28dbc18c71127c5f0.webp)

acme-web

This week: the auth migration, the flaky test, dependencies, the cold start crash, docs, the billing retry, the user service rename and a11y.

On it. I’ll start a session for each one.

Auth migration    Merged
Flaky auth test    Merged
Dependencies    Merged
Cold start crash    Merged
and 4 more

6h 30m
PR    review    merged

flaky auth test
dependencies    auto-merge
cold start crash
auth migration
docs sweep    auto-merge
billing retry
user service
a11y on settings    au[illegible]

### Daily code changes merged into Anthropic’s monorepo

Anthropic went from merging about 2,500 code changes per day into its monorepo in May 2026 to more than 30,000 per day in September 2026, roughly twelve times the May rate.

[1:42](https://www.youtube.com/watch?v=vlfCjWuv4es&t=102s) [An arrow connects 2,500 code changes merged per day in May 2026 to 30,000+ per day in September 2026.](https://assets.ai.engineer/sites/aiecode2025/sha256/1b9162344869576f6946c084748ebcb161896335167cd8a282797e021ea3917d.webp)

code changes merged per day to Anthropic’s monorepo

2,500 → 30,000+
May 2026    September 2026

Redesigning source control at Anthropic, October 2026

### In the loop

A person appears on each of the five cards. Earlier Claude Code guidance put developers in that role throughout a session: they started it, steered the work, decided which commands could run, and checked the output.

[1:58](https://www.youtube.com/watch?v=vlfCjWuv4es&t=118s) [Five illustrated cards show where a person intervenes in a session by steering it, approving actions, saying no, checking the output, or starting the session.](https://assets.ai.engineer/sites/aiecode2025/sha256/1c57c2d68d36c07056f00be7f44245e8add8ee98f0fef850de5a94ff014c5812.webp)

In the loop

Steering it
Approving it
Saying no
Checking it
Starting it

### Out of the loop

When a developer steps back from a task like approving a command or checking output, the model, the harness, or the environment has to take it over.

[2:12](https://www.youtube.com/watch?v=vlfCjWuv4es&t=132s) [The five intervention cards are dimmed. An arrow points down to three labels: the model, the harness, and the environment.](https://assets.ai.engineer/sites/aiecode2025/sha256/cf79d26659828bfeae6d9f2e5b34267714044ca6d93d44d16654b4b26394d076.webp)

Out of the loop

Steering it
Approving it
Saying no
Checking it
Starting it

The model
The harness
The environment

### Code quality and safe execution

When several Claude Code sessions run at once, how can developers check the quality of the code each session writes? If a session runs for hours without supervision, how can they tell it’s running safely?

[2:22](https://www.youtube.com/watch?v=vlfCjWuv4es&t=142s) [On the left, three green check marks accompany “Is the code high-quality?” On the right, a shield with a check mark accompanies “Is it running safely?”](https://assets.ai.engineer/sites/aiecode2025/sha256/a84df97c20f63cb86a4abc374e43f877f48fa63e715fbe51d75aeb630503c14a.webp)

Is the code high-quality?

Is it running safely?

### Manual guidance for a failing authentication test

The user steers Claude toward a code fix using the auth helper, approves edits and npm test, and rejects git reset --hard. The model menu highlights Opus 5.5 while the session still shows Opus 4.5.

[2:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=158s) [Above the timeline, guidance directs the session to fix the code and use the authentication helper. Below it, person icons mark approved edits, an approved test run, and a rejected hard reset. The model menu is open.](https://assets.ai.engineer/sites/aiecode2025/sha256/6997840dba833014112f48d39fda30d3c86b00c9906ee77a2ee891f9194e56d8.webp)

auth.test.ts is failing, can you fix it?

Fix the code, not the test
Use our auth helper
CLAUDE.md
IMPORTANT: always use the auth helper

Timeline:
Read
Edit ✓
Edit ✓
Bash npm test ✓
Edit ✓
Edit ✓
Bash git reset --hard ×
Edit ✓

Interface text:
Fable 5.2    1
Opus 5.5    2
Sonnet 5.5    3
Haiku 5.5    4
Type / for commands
Manual
Opus 4.5

### Auto mode reviews commands during the test fix

Hallie switches the failing-test demo to Auto mode. It allows npm test and blocks git reset --hard without asking her to approve each command. The session then continues with another edit.

[2:56](https://www.youtube.com/watch?v=vlfCjWuv4es&t=176s) [On the tool timeline, a shield with a green check marks the test command as allowed, and a shield with a pink cross marks the hard reset as blocked. Both shields are labeled “auto mode.”](https://assets.ai.engineer/sites/aiecode2025/sha256/8591b5c213b1744758f3d1e4b16cd9ac9d04a14af78f7d33aa034d313ab6cffc.webp)

auth.test.ts is failing, can you fix it?

Timeline:
Read
Edit
Edit
Bash npm test ✓
    auto mode
Edit
Edit
Bash git reset --hard ×
    auto mode
Edit

Interface text:
Type / for commands
Auto
Opus 5.5

### How an auth-helper instruction can send a task off track

Asked to fix auth.test.ts, Claude takes a detour through billing.ts, search.ts and settings.ts. Hallie uses the highlighted CLAUDE.md rule, “always use the auth helper,” to show how emphatic instructions can lead to extra tool calls.

[3:11](https://www.youtube.com/watch?v=vlfCjWuv4es&t=191s) [A prompt asks for a fix to auth.test.ts. A connector links the highlighted auth-helper rule in CLAUDE.md to a path that arcs through billing.ts, search.ts and settings.ts.](https://assets.ai.engineer/sites/aiecode2025/sha256/4d9d35e1cdb3b1b303606cf8e11816fdb9274a47794ad333db927fade9695dc6.webp)

auth.test.ts is failing, can you fix it?

billing.ts
search.ts
settings.ts

CLAUDE.md
1  Think hard and be thorough.
2  Keep going until it’s done.
3  IMPORTANT: always use the auth helper
4  We’re on Node 18.
5  src/legacy still serves our mobile apps

### Documenting a legacy dependency in CLAUDE.md

The highlighted line records that the mobile apps still depend on src/legacy. Hallie recommends keeping that detail in CLAUDE.md because Claude can’t infer the dependency from the code alone.

[3:35](https://www.youtube.com/watch?v=vlfCjWuv4es&t=215s) [A five-line CLAUDE.md excerpt highlights its final line in green. It says that src/legacy still serves the mobile apps.](https://assets.ai.engineer/sites/aiecode2025/sha256/06764c7e260fd10d964d3cdfdf759a5bc9a83e9d65ea3fc9c20c502d3d178097.webp)

CLAUDE.md
1 Think hard and be thorough.
2 Keep going until it’s done.
3 IMPORTANT: always use the auth helper
4 We’re on Node 18.
5 src/legacy still serves our mobile apps

### How auto mode handles project edits and shell commands

While fixing a failing test, Claude reads and edits project files without a check. Auto mode’s classifier allows npm test and blocks git reset --hard because it’s unsure the reset is safe. Claude then continues editing.

[4:10](https://www.youtube.com/watch?v=vlfCjWuv4es&t=250s) [A timeline marks npm test with a green check and git reset --hard with a pink cross. Captions say project reads and edits proceed without a check, commands judged safe are allowed, and uncertain commands are blocked.](https://assets.ai.engineer/sites/aiecode2025/sha256/03813fc8125691b6895016320f6d4f53dd5bc68456bf857fd2cb9896ca4c0efa.webp)

auth.test.ts is failing, can you fix it?

Timeline:
Read
Edit
Edit
Bash npm test ✓
Edit
Edit
Bash git reset --hard ×
Edit
✓

Reads and edits
in your project, no check

Knows it’s safe
✓ allowed

Unsure it’s safe
× blocked

### Configuring auto mode’s environment and permissions

Auto mode’s classifier uses plain-English settings to decide which commands to allow. The example names Acme’s trusted repos, bucket and domains, allows staging deployments, and forbids sending code to third-party APIs.

[4:48](https://www.youtube.com/watch?v=vlfCjWuv4es&t=288s) [A settings.json example highlights entries for trusted repositories, cloud storage and internal domains. An allow rule permits staging deployments, and a hard-deny rule forbids sending code to third-party APIs.](https://assets.ai.engineer/sites/aiecode2025/sha256/7703d56c5aa5c3054ede1ed9acba1796b5de960c2d53fd64d9380a9b8b5f7d63.webp)

~ / .claude / settings.json

{
  "autoMode": {
    "environment": [
      "$defaults",
      "Source control: github.com/acme and all repos under it",
      "Trusted cloud buckets: s3://acme-staging",
      "Trusted internal domains: *.internal.acme.com"
    ],
    "allow": ["$defaults", "Deploying to staging is allowed"],
    "hard_deny": ["$defaults", "Never send code to third-party APIs"]
  }
}

Interface text:
Line numbers: 1 2 3 4 5 6 7 8 9 10 11 12

### Out of the loop

The model steers the work, and auto mode handles approvals. That leaves three decisions for the user: when to say no, how to check the result, and when to start a task.

[5:12](https://www.youtube.com/watch?v=vlfCjWuv4es&t=312s) [Five cards run left to right. The dimmed “Steering it” and “Approving it” cards are labeled “The model” and “Auto mode.” The brighter cards read “Saying no,” “Checking it” and “Starting it.”](https://assets.ai.engineer/sites/aiecode2025/sha256/2c59e4a6d73af5c95bdb4a3ce53f9d71092ff23eeab05eee9a8b29c1745b7af3.webp)

Out of the loop

Steering it
The model

Approving it
Auto mode

Saying no

Checking it

Starting it

### Keeping it safe and verification

When Claude Code runs unattended, two questions remain. How do you keep the run safe? How do you check its work? A shield represents safety, and three green check marks along a path represent verification as the run progresses.

[5:25](https://www.youtube.com/watch?v=vlfCjWuv4es&t=325s) [The “Keeping it safe” panel shows a shield with a check mark over a curved path inside a cloud-labeled box. The “Verification” panel shows three green checks along a path.](https://assets.ai.engineer/sites/aiecode2025/sha256/927da61b4b44cad22c20b6cb837e6321a2f524d12e1c0039c96dbb21bf17d6f6.webp)

Keeping it safe
Verification

### Keeping it safe

The bright shield draws attention to safety first, while verification stays dimmed. With auto mode handling approvals, Hallie asks how to keep Claude Code safe when the model can still stray from instructions.

[5:33](https://www.youtube.com/watch?v=vlfCjWuv4es&t=333s) [A bright shield and check mark sit above “Keeping it safe” on the left. The Verification panel on the right is dimmed.](https://assets.ai.engineer/sites/aiecode2025/sha256/ee6c38499b42abbc53ee524b25c53c9eb3354e19ecbad3a26a42c256dfd4d8d0.webp)

Keeping it safe
Verification

### People caught 14% of dangerous commands and auto mode blocked 89%

In its August 2026 study with 1,053 paid testers, Anthropic swapped in one dangerous command per session. About 14% of testers spotted the command. Auto mode blocked 89% of the dangerous commands.

[5:45](https://www.youtube.com/watch?v=vlfCjWuv4es&t=345s) [Two icon grids compare the reported results. Purple figures mark the 14% caught by people, and white shields mark the 89% blocked by auto mode. The caption describes 1,053 paid testers and one dangerous command per session.](https://assets.ai.engineer/sites/aiecode2025/sha256/ac6bb34f8558ddc5e72f3f6aad3f64a728db73a5a88e5d1cef72fdb79ad6bcd1.webp)

14%
caught by people

89%
blocked by auto mode

1,053 paid testers, one dangerous command swapped in per session. Anthropic, August 2026

### About 1 in 10 dangerous commands still went through

In a study published in August 2026, Anthropic gave 1,053 paid testers one dangerous command per session. Purple figures mark the 14% of testers who caught it. Pink shields mark the roughly 1 in 10 dangerous commands auto mode let through.

[6:00](https://www.youtube.com/watch?v=vlfCjWuv4es&t=360s) [The left grid still shows 14% caught by people. On the right, pink shields among gray shields illustrate the commands that went through.](https://assets.ai.engineer/sites/aiecode2025/sha256/fff4f33b21723851ee0c307fef21bafb9df5596f04e576f0baaf832d23b5def5.webp)

14%
caught by people

1 in 10
still went through

1,053 paid testers, one dangerous command swapped in per session. Anthropic, August 2026

### A deny rule blocks a hard reset before auto mode

Claude Code checks the project’s deny rules first. The Bash call matches the git reset --hard pattern, so Claude Code blocks it before the auto mode classifier sees it.

[6:03](https://www.youtube.com/watch?v=vlfCjWuv4es&t=363s) [An arrow leads from a Git hard-reset tool call to a box labeled “Deny rule” and “blocked.” The auto-mode box is dimmed. Below, the matching permission rule is highlighted in the project settings.](https://assets.ai.engineer/sites/aiecode2025/sha256/ffbba41b8b017b75a3d41ffa511cd5b0df17f9143b6143dd575637f8c02ada40.webp)

Bash git reset --hard
tool call

Deny rule
blocked

Auto mode
never sees it

.claude / settings.json
1  {
2    "permissions": {
3      "deny": ["Bash(git reset --hard *)"]
4    }
5  }

### Can I take it back?

Leave edits to Claude when Git can undo a mistake. Set a project rule when one wrong action could erase uncommitted work, expose a key in the transcript, or send data outside the project.

[6:31](https://www.youtube.com/watch?v=vlfCjWuv4es&t=391s) [The Yes column leaves edits to code, tests and docs, along with anything Git can revert, to the model. The No column calls for a rule when an action deletes something permanently, exposes a secret or sends something outside.](https://assets.ai.engineer/sites/aiecode2025/sha256/21044490166e1957637f087679b80f7a9294063c8426dacaebeabf1b96a24998.webp)

Can I take it back?

Yes
leave it to the model
• Edits to code, tests, docs
• Anything git can revert

No
make it a rule
• Deletes something for good
• Exposes a secret
• Sends something outside

### Blocking access to an environment file

When Claude tries to diagnose a failing authentication test, the demo shows a single deny rule blocking both attempts to open .env: a direct file read and the shell command cat .env.

[6:56](https://www.youtube.com/watch?v=vlfCjWuv4es&t=416s) [Under a request to fix a failing authentication test, a timeline marks a direct file read and a shell read with crosses. A lock marks the environment file in the project tree, and the settings panel highlights its read-denial rule.](https://assets.ai.engineer/sites/aiecode2025/sha256/0b9412fc3a8534a523a5d91a55ec374658ae9324720ab690a51253c1dc2be61a.webp)

auth.test.ts is failing, can you fix it?

Read .env ×
Bash cat .env ×

acme-web
  src
  package.json
  .env
  README.md

.claude / settings.json
{
  "permissions": {
    "deny": [
      "Bash(git reset --hard *)",
      "Read(./.env)"
    ]
  }
}

Interface text:
Line numbers: 1 2 3 4 5 6 7 8

### Enabling the local sandbox

Running npm test loads both the project’s tests and third-party code. With the local sandbox enabled, the operating system limits which files that code can read, which folders it can write to, and which hosts it can reach.

[8:16](https://www.youtube.com/watch?v=vlfCjWuv4es&t=496s) [A settings snippet shows the sandbox enabled beside the /sandbox command. An operating-system boundary surrounds the test process, with captions for file reads, folder writes and network access.](https://assets.ai.engineer/sites/aiecode2025/sha256/1b6308dacd2407e16d6e5244c1798dbf021a230b65c51eaabbf044f9aad4b762.webp)

sandbox
enforced by the operating system

settings.json  "sandbox": { "enabled": true }
/sandbox

auth.test.ts is failing, can you fix it?
Bash  npm test

What npm test runs
vitest run  the script in package.json
src/auth/auth.test.ts  your tests
node_modules/some-package/index.js  code you didn’t write

.env  which files it reads
~/.ssh  which folders it writes
collector.example  which hosts it reaches

### How the sandbox blocks access

When npm test runs a dependency, the operating system enforces sandbox rules on that code too. Here, deny rules block reads of .env and files under ~/.ssh. The sandbox also blocks the connection to the unapproved host collector.example.

[8:37](https://www.youtube.com/watch?v=vlfCjWuv4es&t=517s) [Three paths from dependency code end in crosses at the sandbox boundary. Deny rules protect the environment file and SSH directory, and the destination host lacks approval.](https://assets.ai.engineer/sites/aiecode2025/sha256/2fd33291f62e2283654b40e080e6b501f0293705e167d34be1095a39ff9f0723.webp)

settings.json  "deny": ["Read(./.env)", "Read(~/.ssh/**)"]

Bash  npm test
sandbox    enforced by the operating system

What npm test runs
vitest run    the script in package.json
src/auth/auth.test.ts    your tests
node_modules/some-package/index.js    code you didn’t write

.env    deny rule
~/.ssh    deny rule
collector.example    host not approved

### Running tests on a fresh cloud copy of the repository

The cloud session runs npm test on its own machine using a fresh copy of the repository. You can close your laptop while the tests keep running.

[9:29](https://www.youtube.com/watch?v=vlfCjWuv4es&t=569s) [A cloud outline encloses a timeline marked with the npm test command. Two captions explain that the run uses a fresh repository copy and can continue with the laptop closed.](https://assets.ai.engineer/sites/aiecode2025/sha256/6947ce18d7cfbec8984cc862fdb3059b0cb510644c0dc14adc8165e6eaa84982.webp)

Bash npm test
a fresh copy of our repo
runs with your laptop closed

### Local files and credentials absent from the cloud example

The cloud session runs npm test on a fresh copy of the repository. Hallie keeps her .env file, SSH keys and GitHub token outside Git, so the test runs without them.

[9:41](https://www.youtube.com/watch?v=vlfCjWuv4es&t=581s) [Below the cloud test timeline, three dimmed labels identify an environment file, an SSH directory and a GitHub token as absent.](https://assets.ai.engineer/sites/aiecode2025/sha256/02ab94f4413950180654d7b5b5632de768397df5313f0f97ede818ba01ad0024.webp)

Bash npm test

not in there
.env
~/.ssh
GitHub token

## Full transcript

[0:00](https://www.youtube.com/watch?v=vlfCjWuv4es&t=0s) I had some technical difficulties during

[0:03](https://www.youtube.com/watch?v=vlfCjWuv4es&t=3s) my talk this morning. So, I ran back

[0:05](https://www.youtube.com/watch?v=vlfCjWuv4es&t=5s) home and I'm just going to record it

[0:07](https://www.youtube.com/watch?v=vlfCjWuv4es&t=7s) from my computer instead. So, hi

[0:08](https://www.youtube.com/watch?v=vlfCjWuv4es&t=8s) everyone. I'm Lydia. I work on the clock

[0:10](https://www.youtube.com/watch?v=vlfCjWuv4es&t=10s) code team at Anthropic. And the models

[0:13](https://www.youtube.com/watch?v=vlfCjWuv4es&t=13s) have gotten really good lately. So, we

[0:15](https://www.youtube.com/watch?v=vlfCjWuv4es&t=15s) have Opus 55, which is my favorite daily

[0:17](https://www.youtube.com/watch?v=vlfCjWuv4es&t=17s) driver. And earlier this week, we also

[0:20](https://www.youtube.com/watch?v=vlfCjWuv4es&t=20s) launched Fable 52. And these models are

[0:23](https://www.youtube.com/watch?v=vlfCjWuv4es&t=23s) extremely good at longunning tasks and

[0:25](https://www.youtube.com/watch?v=vlfCjWuv4es&t=25s) understanding your code base. and

[0:27](https://www.youtube.com/watch?v=vlfCjWuv4es&t=27s) especially Fable is also really good at

[0:29](https://www.youtube.com/watch?v=vlfCjWuv4es&t=29s) verifying its own output. But as these

[0:31](https://www.youtube.com/watch?v=vlfCjWuv4es&t=31s) models get better, the way that we work

[0:33](https://www.youtube.com/watch?v=vlfCjWuv4es&t=33s) with them in cloud code also changes. So

[0:36](https://www.youtube.com/watch?v=vlfCjWuv4es&t=36s) earlier this year, I don't know if you

[0:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=38s) remember, but sessions kind of used to

[0:40](https://www.youtube.com/watch?v=vlfCjWuv4es&t=40s) look like this. So we'd use a model like

[0:42](https://www.youtube.com/watch?v=vlfCjWuv4es&t=42s) Opus 45, which honestly for its time was

[0:44](https://www.youtube.com/watch?v=vlfCjWuv4es&t=44s) really good. I really enjoyed working

[0:46](https://www.youtube.com/watch?v=vlfCjWuv4es&t=46s) with Opus45. But compared to today, it

[0:48](https://www.youtube.com/watch?v=vlfCjWuv4es&t=48s) would just stop a lot earlier. So it

[0:50](https://www.youtube.com/watch?v=vlfCjWuv4es&t=50s) would just run for shorter amounts of

[0:52](https://www.youtube.com/watch?v=vlfCjWuv4es&t=52s) time. And we'd also keep correcting it.

[0:55](https://www.youtube.com/watch?v=vlfCjWuv4es&t=55s) We'd keep reminding it to do things, add

[0:57](https://www.youtube.com/watch?v=vlfCjWuv4es&t=57s) things like important, exclamation mark,

[0:59](https://www.youtube.com/watch?v=vlfCjWuv4es&t=59s) all these things to our cloud MD file.

[1:01](https://www.youtube.com/watch?v=vlfCjWuv4es&t=61s) And also features like auto mode didn't

[1:03](https://www.youtube.com/watch?v=vlfCjWuv4es&t=63s) exist yet. So we were either approving

[1:05](https://www.youtube.com/watch?v=vlfCjWuv4es&t=65s) everything manually or taking the

[1:08](https://www.youtube.com/watch?v=vlfCjWuv4es&t=68s) riskier approach with dangerously skip

[1:10](https://www.youtube.com/watch?v=vlfCjWuv4es&t=70s) permission. So we were still very much

[1:13](https://www.youtube.com/watch?v=vlfCjWuv4es&t=73s) in the loop as humans. But as these

[1:16](https://www.youtube.com/watch?v=vlfCjWuv4es&t=76s) models got better at taking on bigger

[1:18](https://www.youtube.com/watch?v=vlfCjWuv4es&t=78s) tasks, our sessions also changed. So

[1:20](https://www.youtube.com/watch?v=vlfCjWuv4es&t=80s) nowadays, at least at Anthropic, our

[1:22](https://www.youtube.com/watch?v=vlfCjWuv4es&t=82s) sessions look more like this. So we're

[1:23](https://www.youtube.com/watch?v=vlfCjWuv4es&t=83s) running dozens of sessions at the same

[1:25](https://www.youtube.com/watch?v=vlfCjWuv4es&t=85s) time, running for hours, verifying their

[1:28](https://www.youtube.com/watch?v=vlfCjWuv4es&t=88s) own results along the way, and we even

[1:30](https://www.youtube.com/watch?v=vlfCjWuv4es&t=90s) have sessions orchestrate other

[1:32](https://www.youtube.com/watch?v=vlfCjWuv4es&t=92s) sessions. So whereas we used to be in

[1:34](https://www.youtube.com/watch?v=vlfCjWuv4es&t=94s) the loop for almost every step and

[1:36](https://www.youtube.com/watch?v=vlfCjWuv4es&t=96s) running one or just a couple sessions at

[1:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=98s) a time, we've now essentially become

[1:40](https://www.youtube.com/watch?v=vlfCjWuv4es&t=100s) managers for dozens of sessions. We also

[1:44](https://www.youtube.com/watch?v=vlfCjWuv4es&t=104s) get a lot more done, though. Now, you

[1:45](https://www.youtube.com/watch?v=vlfCjWuv4es&t=105s) may have seen our blog post in October

[1:47](https://www.youtube.com/watch?v=vlfCjWuv4es&t=107s) about how Enthropic went from about 2500

[1:50](https://www.youtube.com/watch?v=vlfCjWuv4es&t=110s) changes a day in May to more than 30,000

[1:53](https://www.youtube.com/watch?v=vlfCjWuv4es&t=113s) a day in September. But this also means

[1:55](https://www.youtube.com/watch?v=vlfCjWuv4es&t=115s) that we as developers just have to

[1:57](https://www.youtube.com/watch?v=vlfCjWuv4es&t=117s) rethink on what we're focused on

[1:59](https://www.youtube.com/watch?v=vlfCjWuv4es&t=119s) instead. So, if you think about our best

[2:01](https://www.youtube.com/watch?v=vlfCjWuv4es&t=121s) practices from earlier this year, it was

[2:04](https://www.youtube.com/watch?v=vlfCjWuv4es&t=124s) mainly about steering sessions,

[2:05](https://www.youtube.com/watch?v=vlfCjWuv4es&t=125s) approving commands, saying no to certain

[2:08](https://www.youtube.com/watch?v=vlfCjWuv4es&t=128s) things or restricting what cloud could

[2:10](https://www.youtube.com/watch?v=vlfCjWuv4es&t=130s) do, checking the output and also

[2:12](https://www.youtube.com/watch?v=vlfCjWuv4es&t=132s) starting the sessions. So everywhere

[2:14](https://www.youtube.com/watch?v=vlfCjWuv4es&t=134s) that we now step out of that loop,

[2:16](https://www.youtube.com/watch?v=vlfCjWuv4es&t=136s) something else has to take over. And

[2:18](https://www.youtube.com/watch?v=vlfCjWuv4es&t=138s) this is either the model, the harness,

[2:21](https://www.youtube.com/watch?v=vlfCjWuv4es&t=141s) or the environment that it's running in.

[2:23](https://www.youtube.com/watch?v=vlfCjWuv4es&t=143s) So if we are not in a loop to approve

[2:25](https://www.youtube.com/watch?v=vlfCjWuv4es&t=145s) every command, and we're running so many

[2:27](https://www.youtube.com/watch?v=vlfCjWuv4es&t=147s) sessions at the same time, how can we

[2:29](https://www.youtube.com/watch?v=vlfCjWuv4es&t=149s) make sure that the code it's writing is

[2:31](https://www.youtube.com/watch?v=vlfCjWuv4es&t=151s) still high quality? And most

[2:33](https://www.youtube.com/watch?v=vlfCjWuv4es&t=153s) importantly, if these sessions are

[2:34](https://www.youtube.com/watch?v=vlfCjWuv4es&t=154s) running for hours without our

[2:36](https://www.youtube.com/watch?v=vlfCjWuv4es&t=156s) supervision, how can we make sure that

[2:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=158s) they're running safely? So let's go

[2:40](https://www.youtube.com/watch?v=vlfCjWuv4es&t=160s) through that by taking this old session

[2:42](https://www.youtube.com/watch?v=vlfCjWuv4es&t=162s) and seeing what we have to change to get

[2:44](https://www.youtube.com/watch?v=vlfCjWuv4es&t=164s) to that end result that I showed

[2:45](https://www.youtube.com/watch?v=vlfCjWuv4es&t=165s) earlier. Well, first of all, let's use

[2:47](https://www.youtube.com/watch?v=vlfCjWuv4es&t=167s) the actual defaults we start sessions in

[2:49](https://www.youtube.com/watch?v=vlfCjWuv4es&t=169s) nowadays. So that's not really something

[2:51](https://www.youtube.com/watch?v=vlfCjWuv4es&t=171s) you should think about. So let's update

[2:52](https://www.youtube.com/watch?v=vlfCjWuv4es&t=172s) the model here from Opus 45 to Opus 55

[2:55](https://www.youtube.com/watch?v=vlfCjWuv4es&t=175s) and change the manual permission to auto

[2:57](https://www.youtube.com/watch?v=vlfCjWuv4es&t=177s) mode. And Opus 55 is a lot better at

[3:00](https://www.youtube.com/watch?v=vlfCjWuv4es&t=180s) understanding your intent and

[3:02](https://www.youtube.com/watch?v=vlfCjWuv4es&t=182s) understanding your code base. So we

[3:03](https://www.youtube.com/watch?v=vlfCjWuv4es&t=183s) really don't have to steer it as much

[3:04](https://www.youtube.com/watch?v=vlfCjWuv4es&t=184s) anymore midsession. And you also no

[3:07](https://www.youtube.com/watch?v=vlfCjWuv4es&t=187s) longer have to approve every command

[3:08](https://www.youtube.com/watch?v=vlfCjWuv4es&t=188s) because we started making auto mode the

[3:10](https://www.youtube.com/watch?v=vlfCjWuv4es&t=190s) default permission mode back in August.

[3:12](https://www.youtube.com/watch?v=vlfCjWuv4es&t=192s) Now being better at understanding your

[3:14](https://www.youtube.com/watch?v=vlfCjWuv4es&t=194s) intent in your codebase also means that

[3:16](https://www.youtube.com/watch?v=vlfCjWuv4es&t=196s) we have to rethink the instructions we

[3:18](https://www.youtube.com/watch?v=vlfCjWuv4es&t=198s) have in our codebase. So if you followed

[3:21](https://www.youtube.com/watch?v=vlfCjWuv4es&t=201s) our best practices from earlier this

[3:22](https://www.youtube.com/watch?v=vlfCjWuv4es&t=202s) year, you may still have things like

[3:24](https://www.youtube.com/watch?v=vlfCjWuv4es&t=204s) important or never do this exclamation

[3:26](https://www.youtube.com/watch?v=vlfCjWuv4es&t=206s) mark all these things in your cloudmd

[3:28](https://www.youtube.com/watch?v=vlfCjWuv4es&t=208s) file. But with today's models, you

[3:29](https://www.youtube.com/watch?v=vlfCjWuv4es&t=209s) really don't need that and it can

[3:31](https://www.youtube.com/watch?v=vlfCjWuv4es&t=211s) actually make the model go off track. So

[3:33](https://www.youtube.com/watch?v=vlfCjWuv4es&t=213s) you end up with more tool calls and

[3:35](https://www.youtube.com/watch?v=vlfCjWuv4es&t=215s) overall a worse results. So nowadays

[3:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=218s) your cloudmd file should only include

[3:39](https://www.youtube.com/watch?v=vlfCjWuv4es&t=219s) the things that claude cannot see from

[3:41](https://www.youtube.com/watch?v=vlfCjWuv4es&t=221s) your code. So for example, maybe you

[3:43](https://www.youtube.com/watch?v=vlfCjWuv4es&t=223s) have a service that's still using some

[3:45](https://www.youtube.com/watch?v=vlfCjWuv4es&t=225s) legacy source somewhere, but it's not

[3:47](https://www.youtube.com/watch?v=vlfCjWuv4es&t=227s) directly referenced in the code. That's

[3:49](https://www.youtube.com/watch?v=vlfCjWuv4es&t=229s) still good to keep in there. But

[3:51](https://www.youtube.com/watch?v=vlfCjWuv4es&t=231s) everything else like what stack you're

[3:52](https://www.youtube.com/watch?v=vlfCjWuv4es&t=232s) on or where to look, you really don't

[3:54](https://www.youtube.com/watch?v=vlfCjWuv4es&t=234s) need that anymore. And we actually have

[3:56](https://www.youtube.com/watch?v=vlfCjWuv4es&t=236s) a command for this called doctor prompt

[3:58](https://www.youtube.com/watch?v=vlfCjWuv4es&t=238s) audit which makes it really easy to

[4:00](https://www.youtube.com/watch?v=vlfCjWuv4es&t=240s) optimize your environment for a specific

[4:02](https://www.youtube.com/watch?v=vlfCjWuv4es&t=242s) model. So if you're on Opus 55 here and

[4:04](https://www.youtube.com/watch?v=vlfCjWuv4es&t=244s) you run this command, it will

[4:06](https://www.youtube.com/watch?v=vlfCjWuv4es&t=246s) automatically check your CloudMD and

[4:07](https://www.youtube.com/watch?v=vlfCjWuv4es&t=247s) your skills to see what the new model no

[4:10](https://www.youtube.com/watch?v=vlfCjWuv4es&t=250s) longer needs. Now auto mode is a

[4:12](https://www.youtube.com/watch?v=vlfCjWuv4es&t=252s) permission mode that runs a separate

[4:14](https://www.youtube.com/watch?v=vlfCjWuv4es&t=254s) classifier on pretty much everything

[4:16](https://www.youtube.com/watch?v=vlfCjWuv4es&t=256s) that isn't just reading or editing files

[4:18](https://www.youtube.com/watch?v=vlfCjWuv4es&t=258s) in your project. So it automatically

[4:21](https://www.youtube.com/watch?v=vlfCjWuv4es&t=261s) allows commands that it knows are safe,

[4:23](https://www.youtube.com/watch?v=vlfCjWuv4es&t=263s) like running tests or installing

[4:24](https://www.youtube.com/watch?v=vlfCjWuv4es&t=264s) dependencies, but it still blocks

[4:27](https://www.youtube.com/watch?v=vlfCjWuv4es&t=267s) commands where it's unsure whether

[4:28](https://www.youtube.com/watch?v=vlfCjWuv4es&t=268s) they're safe. And normally that's pretty

[4:30](https://www.youtube.com/watch?v=vlfCjWuv4es&t=270s) good. Uh normally that's kind of you

[4:32](https://www.youtube.com/watch?v=vlfCjWuv4es&t=272s) know what we want. But for example here

[4:34](https://www.youtube.com/watch?v=vlfCjWuv4es&t=274s) if we ask it to ship something to

[4:37](https://www.youtube.com/watch?v=vlfCjWuv4es&t=277s) staging we would have just you know

[4:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=278s) approved the push to our deploy repo but

[4:41](https://www.youtube.com/watch?v=vlfCjWuv4es&t=281s) by default auto mode only trusts this

[4:43](https://www.youtube.com/watch?v=vlfCjWuv4es&t=283s) folder and this repo. So it would block

[4:45](https://www.youtube.com/watch?v=vlfCjWuv4es&t=285s) that command. So we're back in the loop.

[4:48](https://www.youtube.com/watch?v=vlfCjWuv4es&t=288s) Now the defaults should be good in most

[4:50](https://www.youtube.com/watch?v=vlfCjWuv4es&t=290s) cases. But if you still notice that you

[4:52](https://www.youtube.com/watch?v=vlfCjWuv4es&t=292s) have to approve a lot of uh commands

[4:54](https://www.youtube.com/watch?v=vlfCjWuv4es&t=294s) even in auto mode. The first thing to

[4:56](https://www.youtube.com/watch?v=vlfCjWuv4es&t=296s) look at is settings here because you can

[4:58](https://www.youtube.com/watch?v=vlfCjWuv4es&t=298s) configure auto mode in your own settings

[5:00](https://www.youtube.com/watch?v=vlfCjWuv4es&t=300s) just in plain English. And this is what

[5:02](https://www.youtube.com/watch?v=vlfCjWuv4es&t=302s) gets sent to the classifier which then

[5:04](https://www.youtube.com/watch?v=vlfCjWuv4es&t=304s) decides whether to reject or allow

[5:05](https://www.youtube.com/watch?v=vlfCjWuv4es&t=305s) certain commands. So here we can just

[5:08](https://www.youtube.com/watch?v=vlfCjWuv4es&t=308s) write which repos, buckets, domains we

[5:10](https://www.youtube.com/watch?v=vlfCjWuv4es&t=310s) trust um and what it should always allow

[5:12](https://www.youtube.com/watch?v=vlfCjWuv4es&t=312s) or never do. Okay, so honestly this was

[5:14](https://www.youtube.com/watch?v=vlfCjWuv4es&t=314s) pretty easy. By just using today's

[5:16](https://www.youtube.com/watch?v=vlfCjWuv4es&t=316s) defaults, we've already gotten rid of

[5:18](https://www.youtube.com/watch?v=vlfCjWuv4es&t=318s) most of our in the loop stuff um because

[5:20](https://www.youtube.com/watch?v=vlfCjWuv4es&t=320s) the model took over the steering and

[5:22](https://www.youtube.com/watch?v=vlfCjWuv4es&t=322s) auto mode took over all the approving.

[5:24](https://www.youtube.com/watch?v=vlfCjWuv4es&t=324s) So getting out of the loop is easy. But

[5:26](https://www.youtube.com/watch?v=vlfCjWuv4es&t=326s) the harder question now is how do we

[5:28](https://www.youtube.com/watch?v=vlfCjWuv4es&t=328s) make sure it's okay that we're not there

[5:30](https://www.youtube.com/watch?v=vlfCjWuv4es&t=330s) anymore. So first is keeping it safe and

[5:32](https://www.youtube.com/watch?v=vlfCjWuv4es&t=332s) then also verification. So let's start

[5:35](https://www.youtube.com/watch?v=vlfCjWuv4es&t=335s) with keeping it safe because everything

[5:36](https://www.youtube.com/watch?v=vlfCjWuv4es&t=336s) so far so upgrading the model and using

[5:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=338s) auto mode is just working with a model.

[5:42](https://www.youtube.com/watch?v=vlfCjWuv4es&t=342s) And in most cases it does what you tell

[5:44](https://www.youtube.com/watch?v=vlfCjWuv4es&t=344s) it to do. But that's not a guarantee. So

[5:47](https://www.youtube.com/watch?v=vlfCjWuv4es&t=347s) in a study we published back in August

[5:49](https://www.youtube.com/watch?v=vlfCjWuv4es&t=349s) we tested auto mode versus real people

[5:50](https://www.youtube.com/watch?v=vlfCjWuv4es&t=350s) in the loop. and only about 14% of

[5:53](https://www.youtube.com/watch?v=vlfCjWuv4es&t=353s) people caught a clearly dangerous

[5:55](https://www.youtube.com/watch?v=vlfCjWuv4es&t=355s) command. Auto mode blocked 89% of them.

[5:58](https://www.youtube.com/watch?v=vlfCjWuv4es&t=358s) So it is a lot better at it than we

[6:00](https://www.youtube.com/watch?v=vlfCjWuv4es&t=360s) were. But about one in 10 of those

[6:03](https://www.youtube.com/watch?v=vlfCjWuv4es&t=363s) commands still went through. So if you

[6:05](https://www.youtube.com/watch?v=vlfCjWuv4es&t=365s) do want the guarantee, you can only

[6:07](https://www.youtube.com/watch?v=vlfCjWuv4es&t=367s) enforce that in the harness. And this is

[6:09](https://www.youtube.com/watch?v=vlfCjWuv4es&t=369s) where some of our older best practices

[6:11](https://www.youtube.com/watch?v=vlfCjWuv4es&t=371s) still matters and that's permissions. So

[6:13](https://www.youtube.com/watch?v=vlfCjWuv4es&t=373s) for example, if you have a deny rule for

[6:15](https://www.youtube.com/watch?v=vlfCjWuv4es&t=375s) a specific tool call in your project,

[6:17](https://www.youtube.com/watch?v=vlfCjWuv4es&t=377s) cloud code looks at this rule before the

[6:19](https://www.youtube.com/watch?v=vlfCjWuv4es&t=379s) automo classifier even looks at it. In

[6:22](https://www.youtube.com/watch?v=vlfCjWuv4es&t=382s) fact, if a tool call matches a deny

[6:24](https://www.youtube.com/watch?v=vlfCjWuv4es&t=384s) rule, the classifier never even sees it.

[6:26](https://www.youtube.com/watch?v=vlfCjWuv4es&t=386s) So having deny rules is still the best

[6:28](https://www.youtube.com/watch?v=vlfCjWuv4es&t=388s) way to make sure clot actually never

[6:30](https://www.youtube.com/watch?v=vlfCjWuv4es&t=390s) does something. But then how do you know

[6:33](https://www.youtube.com/watch?v=vlfCjWuv4es&t=393s) what needs to be a rule in your project

[6:35](https://www.youtube.com/watch?v=vlfCjWuv4es&t=395s) versus what the model can just know

[6:37](https://www.youtube.com/watch?v=vlfCjWuv4es&t=397s) because they've gotten so good at it?

[6:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=398s) Um, I think it kind of just comes down

[6:40](https://www.youtube.com/watch?v=vlfCjWuv4es&t=400s) to if clot gets as wrong once, can I

[6:43](https://www.youtube.com/watch?v=vlfCjWuv4es&t=403s) take it back? So if it writes the wrong

[6:45](https://www.youtube.com/watch?v=vlfCjWuv4es&t=405s) code, you can still revert that. Um, so

[6:47](https://www.youtube.com/watch?v=vlfCjWuv4es&t=407s) you can still leave that to the model,

[6:49](https://www.youtube.com/watch?v=vlfCjWuv4es&t=409s) but if it throws away or uncommitted

[6:51](https://www.youtube.com/watch?v=vlfCjWuv4es&t=411s) changes or it reads a key that's now in

[6:53](https://www.youtube.com/watch?v=vlfCjWuv4es&t=413s) your transcript and you don't don't want

[6:54](https://www.youtube.com/watch?v=vlfCjWuv4es&t=414s) that, that should still be a rule. So

[6:57](https://www.youtube.com/watch?v=vlfCjWuv4es&t=417s) for example, let's say that we just

[6:59](https://www.youtube.com/watch?v=vlfCjWuv4es&t=419s) never want Claude reading our.end file.

[7:01](https://www.youtube.com/watch?v=vlfCjWuv4es&t=421s) Now, by default, auto mode is fine with

[7:03](https://www.youtube.com/watch?v=vlfCjWuv4es&t=423s) this. So it would have just approved it.

[7:05](https://www.youtube.com/watch?v=vlfCjWuv4es&t=425s) So, so if you want to make sure that

[7:07](https://www.youtube.com/watch?v=vlfCjWuv4es&t=427s) claude can never read this file, you

[7:09](https://www.youtube.com/watch?v=vlfCjWuv4es&t=429s) should just add a deny rule. And then

[7:11](https://www.youtube.com/watch?v=vlfCjWuv4es&t=431s) commands like cat cannot open this file

[7:12](https://www.youtube.com/watch?v=vlfCjWuv4es&t=432s) either. But it's not always that

[7:14](https://www.youtube.com/watch?v=vlfCjWuv4es&t=434s) straightforward because sometimes

[7:16](https://www.youtube.com/watch?v=vlfCjWuv4es&t=436s) certain commands can have side effects.

[7:18](https://www.youtube.com/watch?v=vlfCjWuv4es&t=438s) So for example, let's say that npm test

[7:20](https://www.youtube.com/watch?v=vlfCjWuv4es&t=440s) here a very normal command. It's just a

[7:23](https://www.youtube.com/watch?v=vlfCjWuv4es&t=443s) test. So automote runs it. That's fine.

[7:25](https://www.youtube.com/watch?v=vlfCjWuv4es&t=445s) But when the script is running, it runs

[7:27](https://www.youtube.com/watch?v=vlfCjWuv4es&t=447s) code that you didn't write like your

[7:29](https://www.youtube.com/watch?v=vlfCjWuv4es&t=449s) dependencies or maybe claude runs a

[7:31](https://www.youtube.com/watch?v=vlfCjWuv4es&t=451s) script real quick that it wrote just to

[7:33](https://www.youtube.com/watch?v=vlfCjWuv4es&t=453s) quickly check something. So, if one of

[7:35](https://www.youtube.com/watch?v=vlfCjWuv4es&t=455s) those scripts is compromised, it could

[7:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=458s) just open that same end file or your SSH

[7:40](https://www.youtube.com/watch?v=vlfCjWuv4es&t=460s) keys and send those to a server you've

[7:42](https://www.youtube.com/watch?v=vlfCjWuv4es&t=462s) never heard of, right? And even if we

[7:45](https://www.youtube.com/watch?v=vlfCjWuv4es&t=465s) were still there as humans approving

[7:47](https://www.youtube.com/watch?v=vlfCjWuv4es&t=467s) everything, honestly, we would have just

[7:48](https://www.youtube.com/watch?v=vlfCjWuv4es&t=468s) clicked allow because there's nothing to

[7:50](https://www.youtube.com/watch?v=vlfCjWuv4es&t=470s) say no to. It's just part of the script

[7:52](https://www.youtube.com/watch?v=vlfCjWuv4es&t=472s) that in this example case, npm test

[7:55](https://www.youtube.com/watch?v=vlfCjWuv4es&t=475s) runs. And permissions only decide if a

[7:58](https://www.youtube.com/watch?v=vlfCjWuv4es&t=478s) tool call runs, but they cannot limit

[8:00](https://www.youtube.com/watch?v=vlfCjWuv4es&t=480s) what it can reach once it's running. So

[8:03](https://www.youtube.com/watch?v=vlfCjWuv4es&t=483s) we want to make sure that these kinds of

[8:04](https://www.youtube.com/watch?v=vlfCjWuv4es&t=484s) scripts can still run because this is

[8:06](https://www.youtube.com/watch?v=vlfCjWuv4es&t=486s) just part of you know how cloud can

[8:08](https://www.youtube.com/watch?v=vlfCjWuv4es&t=488s) check its own work running tests. So we

[8:11](https://www.youtube.com/watch?v=vlfCjWuv4es&t=491s) cannot just block npm tests but we want

[8:13](https://www.youtube.com/watch?v=vlfCjWuv4es&t=493s) to limit its side effects. So really we

[8:15](https://www.youtube.com/watch?v=vlfCjWuv4es&t=495s) just want to limit the environment that

[8:16](https://www.youtube.com/watch?v=vlfCjWuv4es&t=496s) it's running in. So if you're running it

[8:19](https://www.youtube.com/watch?v=vlfCjWuv4es&t=499s) locally or on your own machine you can

[8:21](https://www.youtube.com/watch?v=vlfCjWuv4es&t=501s) run commands like that in a sandbox. So

[8:24](https://www.youtube.com/watch?v=vlfCjWuv4es&t=504s) then you have control over which files

[8:26](https://www.youtube.com/watch?v=vlfCjWuv4es&t=506s) in a command it can read, which folders

[8:28](https://www.youtube.com/watch?v=vlfCjWuv4es&t=508s) it can write to, uh which hosts it can

[8:30](https://www.youtube.com/watch?v=vlfCjWuv4es&t=510s) reach and the operating system enforces

[8:32](https://www.youtube.com/watch?v=vlfCjWuv4es&t=512s) that. Also this also applies to

[8:35](https://www.youtube.com/watch?v=vlfCjWuv4es&t=515s) everything that command then starts. So

[8:37](https://www.youtube.com/watch?v=vlfCjWuv4es&t=517s) any child processes. So on its own that

[8:40](https://www.youtube.com/watch?v=vlfCjWuv4es&t=520s) deny rule got checked by claude before

[8:42](https://www.youtube.com/watch?v=vlfCjWuv4es&t=522s) the command runs. But all it sees there

[8:44](https://www.youtube.com/watch?v=vlfCjWuv4es&t=524s) is npm test. So it doesn't see which

[8:46](https://www.youtube.com/watch?v=vlfCjWuv4es&t=526s) files that scripts ends up opening. But

[8:48](https://www.youtube.com/watch?v=vlfCjWuv4es&t=528s) with the sandbox on that same rule gets

[8:51](https://www.youtube.com/watch?v=vlfCjWuv4es&t=531s) passed on to the operating system. So

[8:54](https://www.youtube.com/watch?v=vlfCjWuv4es&t=534s) now when when that dependency tries to

[8:56](https://www.youtube.com/watch?v=vlfCjWuv4es&t=536s) open our end file, the operating system

[8:58](https://www.youtube.com/watch?v=vlfCjWuv4es&t=538s) just doesn't let it. And by default, it

[9:00](https://www.youtube.com/watch?v=vlfCjWuv4es&t=540s) can still read most of your machine. So

[9:01](https://www.youtube.com/watch?v=vlfCjWuv4es&t=541s) you'd want to add another rule for your

[9:03](https://www.youtube.com/watch?v=vlfCjWuv4es&t=543s) SSH keys if you care about that. Um, and

[9:06](https://www.youtube.com/watch?v=vlfCjWuv4es&t=546s) it also cannot reach a host that wasn't

[9:09](https://www.youtube.com/watch?v=vlfCjWuv4es&t=549s) approved. But the sandbox only covers

[9:12](https://www.youtube.com/watch?v=vlfCjWuv4es&t=552s) the shell commands that Cloud runs and

[9:13](https://www.youtube.com/watch?v=vlfCjWuv4es&t=553s) whatever those start. But Cloud's own

[9:15](https://www.youtube.com/watch?v=vlfCjWuv4es&t=555s) files, your MCP servers, and all of your

[9:18](https://www.youtube.com/watch?v=vlfCjWuv4es&t=558s) hooks still run outside of it. So the

[9:20](https://www.youtube.com/watch?v=vlfCjWuv4es&t=560s) rest of your session still runs on your

[9:22](https://www.youtube.com/watch?v=vlfCjWuv4es&t=562s) own machine and everything that's on

[9:23](https://www.youtube.com/watch?v=vlfCjWuv4es&t=563s) there. So ideally you want this whole

[9:26](https://www.youtube.com/watch?v=vlfCjWuv4es&t=566s) thing to run somewhere where none of

[9:28](https://www.youtube.com/watch?v=vlfCjWuv4es&t=568s) that is even there in the first place

[9:30](https://www.youtube.com/watch?v=vlfCjWuv4es&t=570s) and that is a cloud session because

[9:32](https://www.youtube.com/watch?v=vlfCjWuv4es&t=572s) there it gets its own machine with a

[9:34](https://www.youtube.com/watch?v=vlfCjWuv4es&t=574s) fresh copy of a repo and this runs even

[9:37](https://www.youtube.com/watch?v=vlfCjWuv4es&t=577s) when we close our laptop which is

[9:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=578s) definitely something you want for these

[9:40](https://www.youtube.com/watch?v=vlfCjWuv4es&t=580s) longer running sessions that run for

[9:42](https://www.youtube.com/watch?v=vlfCjWuv4es&t=582s) hours. So in this case, this

[9:44](https://www.youtube.com/watch?v=vlfCjWuv4es&t=584s) automatically gets rid of the end file

[9:46](https://www.youtube.com/watch?v=vlfCjWuv4es&t=586s) issue because it's not in git. Uh and

[9:48](https://www.youtube.com/watch?v=vlfCjWuv4es&t=588s) neither are my SSH keys or or GitHub

[9:50](https://www.youtube.com/watch?v=vlfCjWuv4es&t=590s) token. And if it still needs an API key,

[9:53](https://www.youtube.com/watch?v=vlfCjWuv4es&t=593s) uh you can add that as a network secret

[9:55](https://www.youtube.com/watch?v=vlfCjWuv4es&t=595s) where it will it will tell you like,

[9:56](https://www.youtube.com/watch?v=vlfCjWuv4es&t=596s) hey, I'm missing an API key midsession.

[9:59](https://www.youtube.com/watch?v=vlfCjWuv4es&t=599s) Um so it can use it without actually

[10:01](https://www.youtube.com/watch?v=vlfCjWuv4es&t=601s) ever sending it to the model. And by

[10:03](https://www.youtube.com/watch?v=vlfCjWuv4es&t=603s) default, also in class cloud session,

[10:05](https://www.youtube.com/watch?v=vlfCjWuv4es&t=605s) network access is limited to things like

[10:08](https://www.youtube.com/watch?v=vlfCjWuv4es&t=608s) package registries and GitHub. So you

[10:10](https://www.youtube.com/watch?v=vlfCjWuv4es&t=610s) have to explicitly add those domains to

[10:12](https://www.youtube.com/watch?v=vlfCjWuv4es&t=612s) your environment for cloud to to reach

[10:14](https://www.youtube.com/watch?v=vlfCjWuv4es&t=614s) other hosts.

[10:16](https://www.youtube.com/watch?v=vlfCjWuv4es&t=616s) Okay. So with a cloud session, we can

[10:18](https://www.youtube.com/watch?v=vlfCjWuv4es&t=618s) run these long running sessions even

[10:20](https://www.youtube.com/watch?v=vlfCjWuv4es&t=620s) with our laptop closed and it keeps us

[10:22](https://www.youtube.com/watch?v=vlfCjWuv4es&t=622s) safe because it runs in its own

[10:24](https://www.youtube.com/watch?v=vlfCjWuv4es&t=624s) environment. Now it's still good to

[10:27](https://www.youtube.com/watch?v=vlfCjWuv4es&t=627s) remember that this machine is

[10:28](https://www.youtube.com/watch?v=vlfCjWuv4es&t=628s) disposable, but the things that leave it

[10:30](https://www.youtube.com/watch?v=vlfCjWuv4es&t=630s) aren't. So a commit pushed or a message

[10:33](https://www.youtube.com/watch?v=vlfCjWuv4es&t=633s) sent through a connector. So we still

[10:36](https://www.youtube.com/watch?v=vlfCjWuv4es&t=636s) run it in auto mode with our deny rules.

[10:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=638s) Just commit it into the repo. So they

[10:39](https://www.youtube.com/watch?v=vlfCjWuv4es&t=639s) come along. Uh so it it could still

[10:41](https://www.youtube.com/watch?v=vlfCjWuv4es&t=641s) catch things like a force push or a

[10:43](https://www.youtube.com/watch?v=vlfCjWuv4es&t=643s) secret in a commit, all that kind of

[10:45](https://www.youtube.com/watch?v=vlfCjWuv4es&t=645s) stuff.

[10:47](https://www.youtube.com/watch?v=vlfCjWuv4es&t=647s) And beyond just safety, um we also need

[10:50](https://www.youtube.com/watch?v=vlfCjWuv4es&t=650s) to make sure that claude is actually

[10:52](https://www.youtube.com/watch?v=vlfCjWuv4es&t=652s) working on the right thing. So next is

[10:54](https://www.youtube.com/watch?v=vlfCjWuv4es&t=654s) verification. And the nice thing about

[10:56](https://www.youtube.com/watch?v=vlfCjWuv4es&t=656s) being in a loop, uh like we kind of know

[10:58](https://www.youtube.com/watch?v=vlfCjWuv4es&t=658s) knew the work was good because we'd

[11:00](https://www.youtube.com/watch?v=vlfCjWuv4es&t=660s) already kind of seen every change. We'd

[11:02](https://www.youtube.com/watch?v=vlfCjWuv4es&t=662s) approved everything. But now cloud works

[11:04](https://www.youtube.com/watch?v=vlfCjWuv4es&t=664s) for hours in the background. We don't

[11:06](https://www.youtube.com/watch?v=vlfCjWuv4es&t=666s) have that anymore. Um, and we're not

[11:08](https://www.youtube.com/watch?v=vlfCjWuv4es&t=668s) really going to read every file at the

[11:10](https://www.youtube.com/watch?v=vlfCjWuv4es&t=670s) end either.

[11:12](https://www.youtube.com/watch?v=vlfCjWuv4es&t=672s) So, you know, say that we ask it to move

[11:14](https://www.youtube.com/watch?v=vlfCjWuv4es&t=674s) our o to a new API. Well, Cloud is very

[11:17](https://www.youtube.com/watch?v=vlfCjWuv4es&t=677s) good at understanding what we want, but

[11:19](https://www.youtube.com/watch?v=vlfCjWuv4es&t=679s) there are still a lot of small decisions

[11:21](https://www.youtube.com/watch?v=vlfCjWuv4es&t=681s) that it has to make along the way. So,

[11:23](https://www.youtube.com/watch?v=vlfCjWuv4es&t=683s) with the exact same prompt, you know,

[11:25](https://www.youtube.com/watch?v=vlfCjWuv4es&t=685s) one run might move it over, but leave

[11:26](https://www.youtube.com/watch?v=vlfCjWuv4es&t=686s) the old API in there, or one might move

[11:29](https://www.youtube.com/watch?v=vlfCjWuv4es&t=689s) it, but only for new login, and another

[11:31](https://www.youtube.com/watch?v=vlfCjWuv4es&t=691s) one might move it, but log everyone out.

[11:34](https://www.youtube.com/watch?v=vlfCjWuv4es&t=694s) And these all look pretty reasonable,

[11:36](https://www.youtube.com/watch?v=vlfCjWuv4es&t=696s) but it really helps to set up these

[11:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=698s) verification points along the way. So

[11:40](https://www.youtube.com/watch?v=vlfCjWuv4es&t=700s) Claude keeps verifying its own results.

[11:43](https://www.youtube.com/watch?v=vlfCjWuv4es&t=703s) And those can be anything that comes

[11:44](https://www.youtube.com/watch?v=vlfCjWuv4es&t=704s) back as a pass or fail. So tests, a

[11:47](https://www.youtube.com/watch?v=vlfCjWuv4es&t=707s) build, a screenshot, whatever you have.

[11:49](https://www.youtube.com/watch?v=vlfCjWuv4es&t=709s) So now we just want to make sure that we

[11:51](https://www.youtube.com/watch?v=vlfCjWuv4es&t=711s) review these verification points. Uh

[11:54](https://www.youtube.com/watch?v=vlfCjWuv4es&t=714s) they're a lot shorter to review than if

[11:55](https://www.youtube.com/watch?v=vlfCjWuv4es&t=715s) we just had to, you know, review the

[11:57](https://www.youtube.com/watch?v=vlfCjWuv4es&t=717s) whole change at the end. So So that's

[11:59](https://www.youtube.com/watch?v=vlfCjWuv4es&t=719s) pretty reasonable. So, if one of these

[12:01](https://www.youtube.com/watch?v=vlfCjWuv4es&t=721s) runs would still log everyone out, it

[12:03](https://www.youtube.com/watch?v=vlfCjWuv4es&t=723s) doesn't get past that first check and

[12:04](https://www.youtube.com/watch?v=vlfCjWuv4es&t=724s) then it just has to find another way.

[12:06](https://www.youtube.com/watch?v=vlfCjWuv4es&t=726s) And there's a couple ways to set this

[12:08](https://www.youtube.com/watch?v=vlfCjWuv4es&t=728s) up. Um, the easiest one is, of course,

[12:10](https://www.youtube.com/watch?v=vlfCjWuv4es&t=730s) just in your prompt. So, you know, you

[12:11](https://www.youtube.com/watch?v=vlfCjWuv4es&t=731s) say what done looks like and ask it to

[12:13](https://www.youtube.com/watch?v=vlfCjWuv4es&t=733s) split that into tasks so they have their

[12:15](https://www.youtube.com/watch?v=vlfCjWuv4es&t=735s) own tests. Um, and I also like to ask it

[12:17](https://www.youtube.com/watch?v=vlfCjWuv4es&t=737s) to show the actual output because you

[12:19](https://www.youtube.com/watch?v=vlfCjWuv4es&t=739s) weren't there to see it. But again,

[12:21](https://www.youtube.com/watch?v=vlfCjWuv4es&t=741s) prompting is not a guarantee that a

[12:23](https://www.youtube.com/watch?v=vlfCjWuv4es&t=743s) model will do it. So for the ones that

[12:25](https://www.youtube.com/watch?v=vlfCjWuv4es&t=745s) have to run guaranteed run, I would use

[12:28](https://www.youtube.com/watch?v=vlfCjWuv4es&t=748s) hooks um which are just you know scripts

[12:30](https://www.youtube.com/watch?v=vlfCjWuv4es&t=750s) that cloud code runs at a specific point

[12:33](https://www.youtube.com/watch?v=vlfCjWuv4es&t=753s) and those give you a lot of control

[12:36](https://www.youtube.com/watch?v=vlfCjWuv4es&t=756s) because you decide when they run and

[12:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=758s) what they run and what they check. So

[12:40](https://www.youtube.com/watch?v=vlfCjWuv4es&t=760s) for example every time it edits a file

[12:42](https://www.youtube.com/watch?v=vlfCjWuv4es&t=762s) we can run our llinter every time it

[12:43](https://www.youtube.com/watch?v=vlfCjWuv4es&t=763s) wants to mark a task as complete we can

[12:46](https://www.youtube.com/watch?v=vlfCjWuv4es&t=766s) run the tests for what it changed. Um,

[12:48](https://www.youtube.com/watch?v=vlfCjWuv4es&t=768s) so you know if it touched our off code

[12:49](https://www.youtube.com/watch?v=vlfCjWuv4es&t=769s) or something it has to pass the off

[12:51](https://www.youtube.com/watch?v=vlfCjWuv4es&t=771s) tests and when it wants to finish the

[12:54](https://www.youtube.com/watch?v=vlfCjWuv4es&t=774s) whole session we can just run the entire

[12:55](https://www.youtube.com/watch?v=vlfCjWuv4es&t=775s) test suite and if one of those fails it

[12:58](https://www.youtube.com/watch?v=vlfCjWuv4es&t=778s) goes oh that's still broken it goes back

[13:00](https://www.youtube.com/watch?v=vlfCjWuv4es&t=780s) until it passes so it fixes it. Now just

[13:04](https://www.youtube.com/watch?v=vlfCjWuv4es&t=784s) one thing um when a test fails by

[13:06](https://www.youtube.com/watch?v=vlfCjWuv4es&t=786s) default most tools they exit with a one

[13:09](https://www.youtube.com/watch?v=vlfCjWuv4es&t=789s) exit code um but with a one that task by

[13:12](https://www.youtube.com/watch?v=vlfCjWuv4es&t=792s) clock code just gets marked as completed

[13:14](https://www.youtube.com/watch?v=vlfCjWuv4es&t=794s) anyway. So, make sure that your scripts

[13:17](https://www.youtube.com/watch?v=vlfCjWuv4es&t=797s) exit with a twoexit code because that's

[13:19](https://www.youtube.com/watch?v=vlfCjWuv4es&t=799s) what actually sends Cloud back to fix

[13:21](https://www.youtube.com/watch?v=vlfCjWuv4es&t=801s) it. So, okay, now we have one session

[13:25](https://www.youtube.com/watch?v=vlfCjWuv4es&t=805s) that I'm pretty comfortable with leaving

[13:27](https://www.youtube.com/watch?v=vlfCjWuv4es&t=807s) alone. So, now we can start with a few

[13:30](https://www.youtube.com/watch?v=vlfCjWuv4es&t=810s) more of those because they've all got

[13:31](https://www.youtube.com/watch?v=vlfCjWuv4es&t=811s) their own machine. So, they're, you

[13:33](https://www.youtube.com/watch?v=vlfCjWuv4es&t=813s) know, not all working in the same folder

[13:35](https://www.youtube.com/watch?v=vlfCjWuv4es&t=815s) and we don't really have to think about

[13:36](https://www.youtube.com/watch?v=vlfCjWuv4es&t=816s) work trees too much ourselves anymore.

[13:39](https://www.youtube.com/watch?v=vlfCjWuv4es&t=819s) But then we still start every session

[13:41](https://www.youtube.com/watch?v=vlfCjWuv4es&t=821s) ourselves and we review every one of

[13:43](https://www.youtube.com/watch?v=vlfCjWuv4es&t=823s) them. And honestly, this is already a

[13:45](https://www.youtube.com/watch?v=vlfCjWuv4es&t=825s) pretty good place to be. We are still

[13:47](https://www.youtube.com/watch?v=vlfCjWuv4es&t=827s) fully in control over what gets started

[13:50](https://www.youtube.com/watch?v=vlfCjWuv4es&t=830s) and when we call a task done. But even

[13:54](https://www.youtube.com/watch?v=vlfCjWuv4es&t=834s) there, um, you know, we're still the

[13:55](https://www.youtube.com/watch?v=vlfCjWuv4es&t=835s) ones kicking off every session and we're

[13:58](https://www.youtube.com/watch?v=vlfCjWuv4es&t=838s) still in the loop for all of them. So, I

[14:00](https://www.youtube.com/watch?v=vlfCjWuv4es&t=840s) think the most important thing to have

[14:01](https://www.youtube.com/watch?v=vlfCjWuv4es&t=841s) in place here now is your CI because all

[14:04](https://www.youtube.com/watch?v=vlfCjWuv4es&t=844s) of these sessions eventually open a PR

[14:06](https://www.youtube.com/watch?v=vlfCjWuv4es&t=846s) and your CI runs the same checks on

[14:08](https://www.youtube.com/watch?v=vlfCjWuv4es&t=848s) every one of them, no matter how that

[14:10](https://www.youtube.com/watch?v=vlfCjWuv4es&t=850s) session was set up or, you know, what

[14:11](https://www.youtube.com/watch?v=vlfCjWuv4es&t=851s) Claude decided to do along the way. And

[14:13](https://www.youtube.com/watch?v=vlfCjWuv4es&t=853s) if you turn on autofix, we have like CI

[14:15](https://www.youtube.com/watch?v=vlfCjWuv4es&t=855s) autofix. Claude can also see when one of

[14:18](https://www.youtube.com/watch?v=vlfCjWuv4es&t=858s) those checks fails and then go back to

[14:20](https://www.youtube.com/watch?v=vlfCjWuv4es&t=860s) fix it. So you're not the one spending

[14:22](https://www.youtube.com/watch?v=vlfCjWuv4es&t=862s) your review on things that you know a

[14:24](https://www.youtube.com/watch?v=vlfCjWuv4es&t=864s) check could have already caught.

[14:26](https://www.youtube.com/watch?v=vlfCjWuv4es&t=866s) And for some of them, you can even turn

[14:28](https://www.youtube.com/watch?v=vlfCjWuv4es&t=868s) on merge when ready. So then GitHub

[14:30](https://www.youtube.com/watch?v=vlfCjWuv4es&t=870s) merges a PR once all of the checks have

[14:33](https://www.youtube.com/watch?v=vlfCjWuv4es&t=873s) passed. So you're not in a loop to

[14:34](https://www.youtube.com/watch?v=vlfCjWuv4es&t=874s) review. Um but again, that that's not

[14:37](https://www.youtube.com/watch?v=vlfCjWuv4es&t=877s) the default and that really depends on

[14:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=878s) how risky the change is. So for example

[14:40](https://www.youtube.com/watch?v=vlfCjWuv4es&t=880s) on our team uh every PR gets reviewed by

[14:43](https://www.youtube.com/watch?v=vlfCjWuv4es&t=883s) cloud first. So not by the session that

[14:45](https://www.youtube.com/watch?v=vlfCjWuv4es&t=885s) wrote it. So it comes with a fresh

[14:48](https://www.youtube.com/watch?v=vlfCjWuv4es&t=888s) context and also a separate group of

[14:50](https://www.youtube.com/watch?v=vlfCjWuv4es&t=890s) agents to just double check every bug it

[14:52](https://www.youtube.com/watch?v=vlfCjWuv4es&t=892s) finds. And then how much we review

[14:55](https://www.youtube.com/watch?v=vlfCjWuv4es&t=895s) ourselves just depends on the risk. So

[14:57](https://www.youtube.com/watch?v=vlfCjWuv4es&t=897s) smaller changes like docer copy, they

[14:59](https://www.youtube.com/watch?v=vlfCjWuv4es&t=899s) don't need a person. So once the

[15:00](https://www.youtube.com/watch?v=vlfCjWuv4es&t=900s) automated checks are green, we're pretty

[15:02](https://www.youtube.com/watch?v=vlfCjWuv4es&t=902s) good to go. um maybe ordinary product

[15:05](https://www.youtube.com/watch?v=vlfCjWuv4es&t=905s) changes just need one reviewer and then

[15:07](https://www.youtube.com/watch?v=vlfCjWuv4es&t=907s) risky changes might need two. Now

[15:10](https://www.youtube.com/watch?v=vlfCjWuv4es&t=910s) ultimately kind of where we see things

[15:12](https://www.youtube.com/watch?v=vlfCjWuv4es&t=912s) going um and we're already kind of doing

[15:15](https://www.youtube.com/watch?v=vlfCjWuv4es&t=915s) more of that at anthropic is that you're

[15:16](https://www.youtube.com/watch?v=vlfCjWuv4es&t=916s) not really starting all of those

[15:18](https://www.youtube.com/watch?v=vlfCjWuv4es&t=918s) sessions yourself either because if

[15:20](https://www.youtube.com/watch?v=vlfCjWuv4es&t=920s) you're anything like me, I'm terrible at

[15:21](https://www.youtube.com/watch?v=vlfCjWuv4es&t=921s) context switching. So having to manage

[15:23](https://www.youtube.com/watch?v=vlfCjWuv4es&t=923s) so many sessions at the at the same time

[15:25](https://www.youtube.com/watch?v=vlfCjWuv4es&t=925s) doesn't always go well. So instead you

[15:28](https://www.youtube.com/watch?v=vlfCjWuv4es&t=928s) just have one main session that's your

[15:30](https://www.youtube.com/watch?v=vlfCjWuv4es&t=930s) orchestrator. And you can already see

[15:32](https://www.youtube.com/watch?v=vlfCjWuv4es&t=932s) this in things like cloud code projects

[15:34](https://www.youtube.com/watch?v=vlfCjWuv4es&t=934s) which is still in beta but you just tell

[15:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=938s) it what it needs to be done and the

[15:40](https://www.youtube.com/watch?v=vlfCjWuv4es&t=940s) orchestrator splits that up and then

[15:42](https://www.youtube.com/watch?v=vlfCjWuv4es&t=942s) starts other sessions for each task and

[15:44](https://www.youtube.com/watch?v=vlfCjWuv4es&t=944s) it keeps track of what comes back. So it

[15:46](https://www.youtube.com/watch?v=vlfCjWuv4es&t=946s) automatically starts and verifies these

[15:48](https://www.youtube.com/watch?v=vlfCjWuv4es&t=948s) threads which are just new sessions. So

[15:51](https://www.youtube.com/watch?v=vlfCjWuv4es&t=951s) instead of talking to eight sessions,

[15:52](https://www.youtube.com/watch?v=vlfCjWuv4es&t=952s) you're just talking to one. And the main

[15:55](https://www.youtube.com/watch?v=vlfCjWuv4es&t=955s) session doesn't write the code itself.

[15:57](https://www.youtube.com/watch?v=vlfCjWuv4es&t=957s) So there you want a model that's really

[15:59](https://www.youtube.com/watch?v=vlfCjWuv4es&t=959s) good at deciding what to do and judging

[16:01](https://www.youtube.com/watch?v=vlfCjWuv4es&t=961s) what comes back. A model like Fable and

[16:04](https://www.youtube.com/watch?v=vlfCjWuv4es&t=964s) if something comes back that isn't

[16:05](https://www.youtube.com/watch?v=vlfCjWuv4es&t=965s) actually done, it just, you know, sends

[16:06](https://www.youtube.com/watch?v=vlfCjWuv4es&t=966s) it back. So until it's actually done and

[16:09](https://www.youtube.com/watch?v=vlfCjWuv4es&t=969s) passes everything and the threads need a

[16:11](https://www.youtube.com/watch?v=vlfCjWuv4es&t=971s) model that's good at actually doing the

[16:13](https://www.youtube.com/watch?v=vlfCjWuv4es&t=973s) work like Opus 55. And you can set those

[16:16](https://www.youtube.com/watch?v=vlfCjWuv4es&t=976s) separately. So you know, one model for

[16:17](https://www.youtube.com/watch?v=vlfCjWuv4es&t=977s) the main session and then another one

[16:19](https://www.youtube.com/watch?v=vlfCjWuv4es&t=979s) for the threads. And another benefit is

[16:22](https://www.youtube.com/watch?v=vlfCjWuv4es&t=982s) that with this setup with code projects

[16:24](https://www.youtube.com/watch?v=vlfCjWuv4es&t=984s) specifically, we don't manage context

[16:26](https://www.youtube.com/watch?v=vlfCjWuv4es&t=986s) anymore. That's all done automatically

[16:28](https://www.youtube.com/watch?v=vlfCjWuv4es&t=988s) for us. So, we no longer have to think

[16:30](https://www.youtube.com/watch?v=vlfCjWuv4es&t=990s) about, you know, when to compact or

[16:32](https://www.youtube.com/watch?v=vlfCjWuv4es&t=992s) clear all of that stuff. Okay. So, if

[16:35](https://www.youtube.com/watch?v=vlfCjWuv4es&t=995s) you look back at those old best

[16:37](https://www.youtube.com/watch?v=vlfCjWuv4es&t=997s) practices, they're all still there, but

[16:38](https://www.youtube.com/watch?v=vlfCjWuv4es&t=998s) they've just moved. So, the model steers

[16:41](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1001s) it now. Um, auto mode approves your deny

[16:44](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1004s) rules and the environment restrict

[16:45](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1005s) what's possible. Um, your test, your

[16:47](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1007s) hooks, and your CI checks it. And then

[16:49](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1009s) ultimately, cloud can just start them as

[16:51](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1011s) well. So almost everything we used to do

[16:54](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1014s) ourselves is now either the model, the

[16:56](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1016s) harness or the environment. And we're

[16:58](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1018s) really just here to start that first one

[17:00](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1020s) and to just review what comes out. So if

[17:03](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1023s) you want to try this in your own setup,

[17:04](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1024s) I would say just take one thing that's

[17:06](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1026s) way bigger than you normally give

[17:08](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1028s) Claude. Make sure your CI actually

[17:10](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1030s) catches the things you care about and

[17:12](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1032s) then spend a bit of time just

[17:13](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1033s) configuring your environment like its

[17:14](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1034s) network access, its secret, and its, you

[17:16](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1036s) know, setup scripts. and then start in

[17:19](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1039s) the cloud and see where your sessions

[17:21](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1041s) get stuck and where you're still in the

[17:22](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1042s) loop because that's the next thing to

[17:24](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1044s) hand over to Claude. I also highly

[17:26](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1046s) recommend checking out our best

[17:28](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1048s) practices in the docs as well. Thank you

[17:30](https://www.youtube.com/watch?v=vlfCjWuv4es&t=1050s) so much.
