# Your Agent Hates Your CLI - Ayush Bhardwaj, Allos AI

Ayush Bhardwaj, Allos AI

[Talk page](https://ai.engineer/talks/rS36lYuvXQ8-your-agent-hates-your-cli) · [Video](https://www.youtube.com/watch?v=rS36lYuvXQ8)

## Your Agent Hates Your CLI

Ayush Bhardwaj examined roughly 20,000 tool calls from his coding sessions. The commands agents wrote immediately afterward showed him which parts of a response they needed, where a failure could disappear, and what to change in his own chat-history CLI.

### One setting, a whole page, 40 lines

[0:00](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=0s)

The agent reviewing Ayush Bhardwaj’s talk needed one setting from a documentation page. It downloaded the page, filtered it and capped the result at 40 lines. In another session, Codex wanted a count of vulnerable dependencies. It requested the whole JSON report and kept the *last* 40 lines, where that report put its totals. Cursor ran a test suite with one failure and kept the last 80 lines. All three cuts worked. Each depended on a guess about where the answer would be.

From the tool author’s side, a successful call can conceal the work it took to use the answer. Sanity’s MCP server had served more than 20,000 agents and handled about 3 million tool calls without a single bug report. Complaints started arriving once Sanity added a tool for giving feedback. Bhardwaj had another way to hear from the users of his CLI. An agent that repeatedly writes the same little program around a command has already left a record of what it wanted.

### How a failed test gets a green status

[1:37](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=97s)

Bhardwaj extracted roughly 20,000 agent calls from his transcripts, covering 14 CLIs including Git, GitHub CLI, Cargo, curl, npm and Docker. A quarter of the calls fed output into filters such as `head`, `grep`, `tail`, Python, `sort` or `wc`. For `git show` and `cargo test`, the share was 62% apiece. The environment running the agent matters to that count. Codex, for example, lets an agent limit output on an individual call, so it has another way to keep a large response out of context.

What succeeded when `cargo test | head` came back green? In Bash with `pipefail` disabled, the pipeline takes the exit status of its last command. `head` can successfully return the requested lines even when `cargo test` fails. In Bhardwaj’s example, a green indicator appeared alongside a compile error. The indicator reported the success of `head`. The diagnostic still told the agent what had happened to the build.

In a second case, the diagnostic itself disappeared. A user reported that a script threw a `ReferenceError` and exited with an error code. `tail` returned three harmless lines. The error never reached the model, and the agent reported success. Bhardwaj’s advice for CLI authors is to put a short failure summary and the path to the full log in the very last lines. An agent reading only the tail can then recognize the failure and retrieve the details.

![A dashed line marks the cutoff for tail -3. The ReferenceError sits above it. Claude saw only the three lines below it and reported success.](https://ai.engineer/api/talk-essay-media/bf0265fa561f801c4ffd8b6af426ce5bfdfb2697e69f2406c78b84914fa46e6b)

The last process in the shell pipeline succeeds, hiding the earlier failure and returning green to the agent.

### Which workarounds belong in chat-history?

[2:59](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=179s)

Bhardwaj’s `chat-history` CLI lets coding agents search earlier sessions across tools. It ships with a skill, an instruction file that explains when and how to call the CLI. An October audit covered about 1,000 calls to this tool. Four out of five times an agent opened an old session, it sent the output straight through a filter or a Python program. That kept irrelevant messages out of context. It also meant writing the filter by hand and guessing again where the relevant message would be.

There’s a good case for leaving a one-off request to `grep` or `head`. The shell already lets the caller choose how much to read. To find candidates for a feature, Bhardwaj grouped pipes across the broader CLI sample by CLI subcommand and downstream tools. Arguments could vary. On that definition, 83% of pipes belonged to patterns seen at least ten times. The search term or line count could change on every call while the agent kept asking for the same operation.

### A message window, fewer fields and a batch request

[4:32](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=272s)

Opening one long session in full produced over a million characters. Reading the part around a search match took about 3,000. A harness can cap either response at a fixed size. Asking for the messages around a match tells the CLI which part of the session deserves that space. Two recurring filters suggested changes to what `chat-history` returned.

- **Select messages with `view`.** Agents searched and sliced a session after opening it. The `view` command gained options to return the messages around a match, only the matching messages, or the last few messages under a size cap. An agent looking for an earlier decision could request the surrounding discussion directly.

- **Select fields with compact `search`.** Agents ran Python over JSON search results to extract the few fields they wanted. Compact search returned one line per hit. It preserved the same set of hits in about one-sixth the space of the JSON output.

What did a `for` loop around `inspect` reveal? Each invocation summarized one session, and about a third of `inspect` calls came from agents looping over several sessions. The summary was already the operation they wanted. The loop supplied the missing ability to ask about several sessions at once. Bhardwaj changed `inspect` to accept them in one call.

A cloud-cost CLI team found agents writing `jq` and Python pipelines over its JSON dumps. The team moved filtering into the CLI, added a compact format and wrote a skill to teach both. In its benchmark, the agent went from answering zero of six hard questions correctly to answering all six. Cost across the benchmark fell 41%. The team tested those changes together, so the result applies to the combination. One of the authors likened reading the logs to watching usability tests.

![Three rows pair shell pipelines with CLI features for the same tasks: limiting a session view, returning one compact line per search hit, and inspecting several sessions in one brief call.](https://ai.engineer/api/talk-essay-media/aa0f8be884d7edbdcf179da97e5e7cf4ed930c389f8bf9922eea642894a26daf)

The redesigned commands expose the narrow reads and batch operations that agents had been assembling from pipes.

### Why 95 searches passed a flag that did nothing

[6:33](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=393s)

A change to `chat-history` made its deep-search flag a no-op. Bhardwaj deliberately left the instruction to use it in the skill. All 95 searches passed the flag. Then two sessions loaded a rewritten skill that dropped the instruction. Across those two sessions, no search passed the obsolete flag, and both agents tried the new options on their first read.

For an agent choosing its next command, `SKILL.md`, help text and error messages are all parts of the interface. Stale instructions can keep generating obsolete calls long after the implementation changes. Once agents learned the new flags, nearly every search used compact output and nearly every read used the new options. The share of session reads that went through a pipe fell from 82% to about a quarter.

### Replay the command and check what the agent used next

[7:36](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=456s)

A recorded command supplies an input for a replay test. The next action supplies a rough answer key. Suppose an agent searched old sessions and then opened one of the hits. Replay that search against the old and new builds. If the change affects ranking, track where the session the agent actually used appears in each result. The search can complete successfully in both builds while the useful hit slips down the list.

For a format change, check whether the new output still contains the information used downstream. For a new default, replay the same arguments and identify which calls change behavior. An immediate retry with different arguments is a candidate failure to investigate. It gives a clue to the agent’s intent, though the agent may have other reasons for changing the query. Keep a few cases out of the design work and use them to score the fix.

### Small defaults that tell the caller how to get more

[8:20](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=500s)

Starting over, Bhardwaj would build ranges and limits first. In his broader audit, 55% of the measured pipelines used only simple downstream tools for slicing output, such as `head` and `tail`. Adoption varied even where a CLI already offered a shortcut. Agents used the limit flag on 81% of the GitHub CLI list commands he observed. Cargo’s quiet test mode prints a character per test and still reports failures, yet agents requested it on only about one run in ten and often piped the output themselves.

Teach the flag in the skill and help text, and let the output itself teach the next call. A default page can tell the caller it is seeing 20 of 300 results and can request the next page with an offset of 20. The count tells the agent its view is partial. The offset gives it the argument it needs to continue.

### What a 59% reduction in tool output bought

[9:20](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=560s)

An agent mode can bundle several of these defaults. Some coding agents set a shell environment variable that lets a CLI recognize its caller. In Bhardwaj’s database CLI example, a person at a terminal gets a formatted table. An agent gets a plain table that says how many rows were left out and how to retrieve them. A very large query stops early and reports that it stopped. Errors come back as JSON. The omitted-row count and early-stop message tell the agent how complete a result it has.

The database CLI cut the output the model read from the tool by 59%. Across everything the model read, the saving amounted to about 0.5% of input tokens. Cost and runtime showed no improvement, and the agent was already getting the answers right. The cloud-cost benchmark had room to improve on correctness, moving from zero to six correct answers on its hard questions. To tell what changed for the user, count turns and correct answers alongside tokens, cost and elapsed time.

![DuckDB agent mode reduced output by 59%. A narrow pink segment on a bar representing all tokens the model read shows that the savings amounted to about 0.5% of that total.](https://ai.engineer/api/talk-essay-media/cfa06401a77a6ad5036ed3bdf0d9ca4c87f126aad45373db689ddc49f61993ad)

The 59% saving applies to tool output. The narrow strip shows how little of the agent’s overall token use that represented.

### Give the agent a design decision it can test

[10:14](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=614s)

Why not ask the agent to design its own CLI in one shot? In Bhardwaj’s experience, an open-ended request tends to produce a familiar human-facing tool with agent support added afterward. He cited a study in which most official MCP servers wrapped an existing API, and a separate check that flagged issues in 97% of the tool descriptions it examined. Generating more instructions has costs of its own. In a study of agent-written context files, the files increased the cost of running tasks without improving task success and performed worse than files people wrote.

Give the agent a transcript and a test it can run, and the assignment becomes concrete. A tool author can decide that a session reader ought to return the discussion around a match. The agent can replay the `view` call, check whether the returned window includes the message used next, and try a correction if the useful passage falls outside it.

Start by backing up the transcripts. Bhardwaj noted that inactive sessions in one coding tool could be deleted after 30 days. For a CLI, search for its name in commands that feed a pipe and read what the downstream program keeps. For an MCP server, look for two calls to the same tool in quick succession and compare their arguments. The changed argument may be the clearest account you have of what the first response left unresolved. To find those calls across past coding sessions, Bhardwaj uses [chat-history](https://github.com/ay-bh/chat-history).

## Key takeaways

- Group pipes by CLI subcommand and downstream tools, allowing the arguments to vary. Repeated filters revealed demand for selecting messages and trimming fields. Loops around `inspect` revealed demand for several session summaries in one call.
- Check which command supplied the pipeline’s exit status and which lines reached the model. A short final diagnostic and a path to the full log give an agent reading only the tail a way to recognize and investigate failure.
- Test the instructions along with the executable. An obsolete flag appeared in 95 of 95 searches while the skill still taught it. Both sessions that loaded the rewritten skill tried the new flags on their first read.
- Reserve real calls for evaluation. Check whether the new build still returns what the agent went on to use, then measure whether agents reach correct answers in fewer turns.

## Slides

### Your agent hates your CLI

Ayush Bhardwaj built a CLI so agents could reuse context from past sessions across tools. He opens with Claude Code downloading a whole documentation page to find one setting, then keeping 40 lines of output.

[0:00](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=0s) [The title “Your agent hates your CLI” appears in white and yellow. The byline identifies Ayush Bhardwaj as Tech Lead at Allos.](https://assets.ai.engineer/sites/aiecode2025/sha256/af96b58e573ac44e71f16b6cfeda7bc5de0d7b2416fbc049f8d71edf4760c379.webp)

Your agent
hates your CLI

Ayush Bhardwaj
Tech Lead @ Allos
Prev: D. E. Shaw & Co, Arista Networks

docker logs --since 5m api-1 2>&1 | rg -i -C 3 'error'
curl -sL …/settings-reference.md | grep -n -A3 "cleanupPe[illegible]ay[illegible]
python3 -c '…' | grep -iE | awk | head -c 80000 | head -80 | rg -n | tail -n 40 | sed -n '1,[illegible]

### Agents work around friction without filing bug reports.

By June 2026, more than 20,000 agents had made about 3 million tool calls on Sanity’s MCP server since launch without filing a single bug report. When Sanity added a feedback tool, agents began describing what went wrong.

[1:03](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=63s) [Beside the headline, a June 2026 Sanity engineering blog excerpt says agents are among Sanity’s heaviest users and highlights that none of them report bugs.](https://assets.ai.engineer/sites/aiecode2025/sha256/c168d4dda05ce7a2e6eae7f9fb62a1f4be52a684fbcceed6e6f1c385f6a68a4d.webp)

Agents work around friction without filing bug reports.

Agents are some of Sanity's heaviest users, but none of them report bugs. Here's how we've started gathering feedback from agents, and how you can do it too.

Sanity engineering blog, June 2026

### Failed runs came back green.

Without pipefail, Bash used the exit code from head, the last command in the pipeline. The tool showed a green success label even as the cargo test output said compilation failed.

[2:06](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=126s) [A filtered cargo test command carries a green “tool: success” label. Red compiler output directly below says a function received four of its five required arguments and compilation failed.](https://assets.ai.engineer/sites/aiecode2025/sha256/6c3713315a0e47cb8830910d931520ecc9095f0fc400810b10214e893564ee4c.webp)

Failed runs came back green.

Bash(cargo test 2>&1
| grep -E 'calling_session|^test result|FAILED|panicked|^error'
| head -30)  tool: success
  └ error[E0061]: this function takes 5 arguments
    but 4 arguments were supplied
    error: could not compile `chat-history`
    (test "search_bm25") due to 1 previous error

Interface text:
claude — 27 Oct

### The error never reached the model

In Claude Code issue #39945, generate-files.js threw a ReferenceError. The error appeared before the final three lines, so tail -3 discarded it. Claude saw only those three lines and reported success.

[2:30](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=150s) [A dashed line marks the cutoff for tail -3. The ReferenceError sits above it. Claude saw only the three lines below it and reported success.](https://assets.ai.engineer/sites/aiecode2025/sha256/bf0265fa561f801c4ffd8b6af426ce5bfdfb2697e69f2406c78b84914fa46e6b.webp)

The error never reached the model.

node generate-files.js 2>&1 | tail -3

ReferenceError
thrown away unread

tail -3 keeps only what's below

all Claude saw
reported success

Claude Code issue #39945. Diagram of the reported failure.

### Agents piped 80% of reads of old sessions

In the October audit, agents filtered the output of 188 of 236 view calls to old sessions. Each pink square marks a call where the agent used a command such as grep or tail to narrow what it read.

[3:31](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=211s) [Each square represents one view call in the October audit. Pink marks the 188 of 236 calls that agents piped to shorten the output. Dark squares mark raw reads.](https://assets.ai.engineer/sites/aiecode2025/sha256/9d18ae67310453e3b7c3a41e9dbd87cd180fba9aa17df994cda9844fb4955e6f.webp)

80% of agents' reads of old sessions were piped through something to cut them down.

piped   read raw · one square = one view call

what the reads were piped into
| head
| rg
| tail
| grep
| sed
| cut
| python3
| awk

188 of 236 view calls, October audit.

### A full session read takes 1.2 million characters

One session contained 1,214,443 characters. Selecting the passage around the search match returned just 3,159 characters. A size cap limits how much the agent reads. Selection lets it choose which messages it reads.

[4:31](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=271s) [A long pink bar represents the full session at 1,214,443 characters. A thin green mark represents the excerpt around the match at 3,159 characters.](https://assets.ai.engineer/sites/aiecode2025/sha256/9e0c1279de731d6ea10abfe44a249c85e4adad56d132752f54334f71dd28291d.webp)

A full read: 1.2 million characters

One session, two ways
whole session   1,214,443 chars →
just the part around the match   3,159 chars

### First, build what the pipes ask for

Agents’ shell commands showed Bhardwaj what to build into his CLI. Agents can now read around a matching message, get the same search hits in about one-sixth the space of JSON, and inspect several sessions in one call.

[4:57](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=297s) [Three rows pair shell pipelines with CLI features for the same tasks: limiting a session view, returning one compact line per search hit, and inspecting several sessions in one brief call.](https://assets.ai.engineer/sites/aiecode2025/sha256/aa0f8be884d7edbdcf179da97e5e7cf4ed930c389f8bf9922eea642894a26daf.webp)

First, build what the pipes ask for

view <id> --plain | rg -n "…" | head -60
view <id> --plain | sed -n '7970,8040p'
→ view <id> --around 42 · --grep "…" · --tail 6 · --max-chars 1500
Hits carry a message number for the next call.

search "…" --deep --json | python3 -c "…for r in d['results']…"
→ search "…" --compact
One line per hit. ~6× smaller than JSON, same hits.

for id in …; do chat-history inspect $id | head -45; done
→ inspect a b c --brief
One call instead of a shell loop.

Glenn Gillen, "We cut Claude's token usage 79% by redesigning our CLI for agents", infracost.io blog, May 2026.

### Then teach it in your docs

Agents passed --deep in all 95 searches because SKILL.md still told them to, even though the flag did nothing. After the instruction was removed, 0 of 9 searches used it. Both sessions tried the new flags on their first read.

[6:32](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=392s) [All 95 searches used --deep while the skill recommended it. After that instruction was removed, 0 of 9 used it. Both sessions tried the new flags on their first read.](https://assets.ai.engineer/sites/aiecode2025/sha256/22c228de27636d75e7d30e13ed1394f028d47cea2f19e8af1410d55ae3b73014.webp)

Then teach it in your docs

95 / 95
searches passed --deep. The skill still said "always use it", and I knew it was a no-op.

0 / 9
once the skill stopped saying it. Both sessions tried the new flags on their first read.

Your SKILL.md, --help and examples are the interface.

### After the interface change

Around 19 Oct, agents began using selection and size flags on nearly every read. From 30 Oct–7 Nov, 99% of searches used --compact, and agents piped 25% of reads, down from 82% in March–September.

[7:17](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=437s) [The chart shows compact searches and selection or size flags rising sharply around October 19. The summary reports that 99% of searches used --compact and that piped reads fell from 82% to 25%.](https://assets.ai.engineer/sites/aiecode2025/sha256/3eb4a247ddc39acd0161e3e2b02587ff3d6fd9afe1c329af30c3d413107bc439.webp)

After the interface change

Y-axis ticks: 100%, 50%, 0%
Timeline: 7 Sep, 14, 21, 28, 5 Oct, 12, 19, 26, 2 Nov

new flags reach agents
search --compact
selection / size flags
view piped

searches now using --compact
99%

reads piped
82% → 25%

Weekly share of calls. Right: March–September vs 30 Oct–7 Nov.

### Your transcripts are your test cases

Replay recorded commands on the old and new builds. The agent’s next action gives you a rough answer key: does the result it chose still rank first? A retry with different arguments points to a likely failure.

[7:34](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=454s) [Replay recorded commands on old and new builds, using the agent’s next action as a rough answer key. Check whether the chosen result stays on top, whether output goes missing, and which commands change behavior. A retry with different arguments flags a possible failure.](https://assets.ai.engineer/sites/aiecode2025/sha256/2c9945142ee7bc7b834fc8cd55a08ed7e6d859abc78d6dcc4316070fd86510f9.webp)

Your transcripts are your test cases

Commands agents ran
your test cases

replay →
old build
new build
compare →

What the agent did next
your rough answer key

New ranking
Does the result the agent used stay on top?

New output format
Did anything go missing?

New default
Which real commands now behave differently?

A retry with different arguments is probably a failing case.

### Build these first

In Bhardwaj’s saved transcripts, 55% of measured pipelines used only head, tail or sed downstream, and 81% of gh … list calls used --limit. He recommends ranges and limits first. Print one page by default and show how to get the next.

[8:18](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=498s) [55% of measured pipelines used only head, tail or sed, and 81% of gh list calls used --limit. The slide recommends ranges, limits and bounded output, with an example showing 20 of 300 results and the offset for the next page.](https://assets.ai.engineer/sites/aiecode2025/sha256/4304224a20780034ee1fe4e130c8fbfe29f87795038702fba897cd4d6b909ebc.webp)

Build these first

55% of measured pipelines used only head, tail or sed.
Ship ranges and limits first.

81% of gh … list calls used --limit.
A flag only helps if agents reach for it.

Print less
Bound big outputs by default.
End with:
showing 20 of 300, add --offset 20

My saved transcripts.

### Agent mode cut output 59%.

In DuckDB’s benchmark, agent mode cut CLI output by 59%. The thin pink segment shows how much that saved across all the tokens the model read: about 0.5%. Neither cost nor runtime improved.

[9:18](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=558s) [DuckDB agent mode reduced output by 59%. A narrow pink segment on a bar representing all tokens the model read shows that the savings amounted to about 0.5% of that total.](https://assets.ai.engineer/sites/aiecode2025/sha256/cfa06401a77a6ad5036ed3bdf0d9ca4c87f126aad45373db689ddc49f61993ad.webp)

Agent mode cut output 59%.

All the tokens the model read
≈0.5% is what that 59% saved.

Infracost's agents started getting answers right. DuckDB's already were.

To detect one: an agent env var like CLAUDECODE and no TTY. It's only a hint, so always offer an override.

DuckDB agent mode benchmark, Oct 2026. Infracost, May 2026.

### Agents can't design this for you yet

Ask an agent to design a CLI for another agent and it tends to picture a human user. Decide what the tool should do, then give the agent session transcripts to spot recurring failures and a test it can run to check its fixes.

[10:12](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=612s) [The left column lists three tasks agents struggle with: designing for agents, knowing what works, and writing their own docs. The right column lists two strengths: finding problems across transcripts and improving a tool against a test.](https://assets.ai.engineer/sites/aiecode2025/sha256/df60282de7205833d9dab8672e37c5a746f15c9131779580fa6ba131a0d750b1.webp)

Agents can't design this for you yet

Not good at yet

Designing for agents
They picture a person.

Knowing what works
They copy what's common, and most of it is poor.

Writing their own docs
Agent-written context files didn't help tasks succeed.

Already good at

Reading your transcripts
Finding what's going wrong across many sessions.

Improving a tool against a test
Sometimes past what people wrote by hand.

You decide what the tool should be. Agents check it against what really happened.

"From REST to MCP" (arXiv 2507.16044). "MCP Tool Descriptions Are Smelly!" (arXiv 2602.14878). "Evaluating AGENTS.md" (ETH Zurich, arXiv 2602.11988). Anthropic, "Writing effective tools for agents" (2025).

### Your agents' transcripts are the product analytics you aren't reading.

Read agent transcripts as product analytics. A pipe shows what an agent does with your CLI’s output. An immediate second call with different arguments shows how it adjusts its request.

[11:52](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=712s) [Below “If you remember one thing,” the slide reads “Your agents' transcripts are the product analytics you aren't reading.” The words “product analytics” appear in yellow.](https://assets.ai.engineer/sites/aiecode2025/sha256/ca6c2544c18d172eacd68851a25eff5ac6d770fdc8bc7e37b83e89f6ed4b5270.webp)

If you remember one thing:

Your agents' transcripts are the product analytics you aren't reading.

Ayush Bhardwaj · Your agent hates your CLI

### The workaround is the bug report.

One agent uses grep to find a setting. Two others use tail to trim audit and test output. Bhardwaj labels each workaround a bug report and points to the chat-history CLI for reading your own agents’ transcripts.

[12:04](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=724s) [Three terminal excerpts show a settings lookup, a dependency audit, and a failed test run. Each carries a green “BUG REPORT” label. Below them, “Go read your transcripts” appears above the link to the talk’s CLI.](https://assets.ai.engineer/sites/aiecode2025/sha256/70a39978f8dc7c3eb8d1bab66f39a7be995a482d54acf463d3e8ce28bc99253b.webp)

The workaround is the bug report.

● Bash(curl -sL …/settings-reference.md
| grep -n -i -A3 "cleanupPeriodDays" | head -40 [illegible])
  ⎿  64:| [`cleanupPeriodDays`](#cleanupperioddays) | Choose how many days…
     … +34 lines
BUG REPORT

• Ran npm audit --omit=dev --json 2>&1 | tail -40
  ⎿ …
      "high": 9,
      "critical": 3,
      "total": 14
BUG REPORT

Running the library tests to check the parser change.
cargo test --lib 2>&1 | tail -80
test session::tests::parse_codex_jsonl_with_fixture ... FAILED
test result: FAILED. 145 passed; 1 failed; 2 ignored
BUG REPORT

Go read your transcripts.
CLI from this talk: github.com/ay-bh/chat-history

Interface text:
claude — ~/talk
codex — ~/Documents/GitHub/data-management
Cursor — chat-history · Agent
Exit code 101
last 80 lines kept

## Full transcript

[0:00](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=0s) Hi, I am Aush. I built a CLI that lets

[0:03](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=3s) your agent reuse context from your past

[0:05](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=5s) session across tools. While improving

[0:07](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=7s) it, I realized why my agent secretly

[0:09](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=9s) hated my CLI and what you can do about

[0:12](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=12s) yours.

[0:15](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=15s) Cloud Code wanted one setting from a

[0:17](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=17s) docs page. So, it downloaded the whole

[0:19](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=19s) page, grabbed it, and kept the result at

[0:22](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=22s) 40 lines.

[0:24](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=24s) Codex wanted to know how many

[0:26](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=26s) dependencies were vulnerable. It asked

[0:28](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=28s) the whole JSON report and kept the last

[0:30](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=30s) 40 lines where this report puts the

[0:32](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=32s) totals.

[0:34](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=34s) Cursor ran a test suit, found one

[0:36](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=36s) failure and kept the last 80 lines. All

[0:39](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=39s) three cuts kept the information shown

[0:41](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=41s) here. Each one coded in an assumption

[0:44](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=44s) about where the answer sits in the

[0:45](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=45s) output that can hide friction from the

[0:47](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=47s) people who build these tools. One pipe

[0:50](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=50s) tells you very little. The same pipe

[0:52](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=52s) written again and again can point to

[0:54](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=54s) something your tool is missing. These

[0:56](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=56s) are real calls from my own transcripts

[0:58](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=58s) and the cloud code one was the agent

[1:00](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=60s) reviewing this talk and it still

[1:01](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=61s) couldn't help itself.

[1:05](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=65s) Agents can work around friction without

[1:07](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=67s) filing a bug report. They write a little

[1:10](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=70s) program around your program. In June,

[1:13](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=73s) Sanity, the content platform, wrote up

[1:16](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=76s) their MCP server. More than 20,000

[1:18](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=78s) agents, about 3 million tool calls since

[1:21](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=81s) launch and not a single bug report. So

[1:23](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=83s) they shipped a give feedback tool. Once

[1:25](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=85s) agent had somewhere to complain, they

[1:28](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=88s) started expressing what's wrong. But you

[1:30](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=90s) don't need a feedback tool to start.

[1:32](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=92s) Feedback is already sitting somewhere in

[1:34](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=94s) your transcripts.

[1:37](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=97s) I I extracted about 20,000 agent calls

[1:40](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=100s) from my same transcripts. 14 CLI appear

[1:42](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=102s) in that sample. Git, GitHub CLI, cargo,

[1:45](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=105s) curl, npm, docker, and others. A quarter

[1:48](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=108s) fed output into filters such as head,

[1:50](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=110s) grab, tail, python, sort or wc. For read

[1:54](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=114s) heavy commands, it's more than half. Git

[1:56](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=116s) show and cargo test 62% each. Some

[1:59](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=119s) harnesses pipe far less. Codeex for

[2:02](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=122s) example lets the agent set an output

[2:05](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=125s) limit on each call.

[2:08](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=128s) And pipes don't only cut noise in batch

[2:11](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=131s) without pipe fail. A pipe then returns

[2:13](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=133s) the exit code of its last command. So

[2:15](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=135s) let's say you pipe cargo test into head.

[2:18](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=138s) It can report success even when the test

[2:20](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=140s) failed. Here's one example on the

[2:22](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=142s) screen. A compile error and a green dot.

[2:27](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=147s) The status was wrong but the agent could

[2:30](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=150s) still read the error.

[2:32](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=152s) A pipe can also cut the error itself

[2:35](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=155s) which is worse. In a cloud code issue on

[2:37](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=157s) GitHub, a user reported it a script

[2:39](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=159s) through a reference error and exited

[2:41](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=161s) with an error code. tail showed three

[2:43](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=163s) harmless lines and Claude reported

[2:45](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=165s) success. The error never reached the

[2:48](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=168s) model. So my advice would be to fail

[2:50](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=170s) loudly when a tool fails. Make the very

[2:53](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=173s) last lines a short summary and a part to

[2:56](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=176s) the full log.

[2:59](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=179s) I learned all this while tinkering with

[3:01](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=181s) a CLI I built. It's a small tool that

[3:04](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=184s) lets coding agents search their own past

[3:06](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=186s) sessions across cloud code, codeex,

[3:09](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=189s) cursor. It ships with a skill so the

[3:11](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=191s) agent can decide when and how to call

[3:13](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=193s) it. Agents are the users that I'm

[3:16](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=196s) studying. My October audit recorded

[3:18](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=198s) about thousand calls and that's my small

[3:21](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=201s) case study. The method works on anything

[3:23](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=203s) your agents calls. Just look at what

[3:26](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=206s) they do right after the call.

[3:30](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=210s) 80%.

[3:31](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=211s) Four out of five times an agent opened

[3:33](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=213s) an old session with my tool. The output

[3:35](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=215s) went straight into crab sad head tail or

[3:38](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=218s) Python. I call that the pipe text. A

[3:41](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=221s) pipe keeps junk out of context and

[3:43](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=223s) that's fine. The text is that the agent

[3:46](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=226s) writing that same cut down command by

[3:48](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=228s) hand every single time. It's small each

[3:51](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=231s) time and it mostly works but each one is

[3:53](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=233s) another guess about where the answer is.

[3:58](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=238s) The fair objection comes from a hacker

[3:59](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=239s) news thread about designing CLS for

[4:02](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=242s) agent that I was reading. The command

[4:04](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=244s) says to leave it to the agents to choose

[4:07](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=247s) how much output to reads. agent can pipe

[4:09](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=249s) through grap or head perfectly well.

[4:12](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=252s) Well, they can and for oneoff they

[4:14](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=254s) should. I'm not against that. But across

[4:17](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=257s) those everyday CLIs, I found 83% of

[4:19](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=259s) pipes fell into the patterns seen at

[4:22](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=262s) least 10 times. The same CLI subcomand

[4:24](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=264s) and pipe tools. The arguments passed

[4:26](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=266s) could differ, but the agent keep asking

[4:29](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=269s) for the same things.

[4:32](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=272s) I [snorts] understand the output they're

[4:34](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=274s) cutting is probably big. I opened one

[4:36](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=276s) long session with my tool two ways the

[4:39](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=279s) whole thing 1.2 2 million characters and

[4:42](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=282s) that's why the agent pipe just the part

[4:44](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=284s) around the search match was about 3,000

[4:46](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=286s) characters. Harnesses do cap output but

[4:49](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=289s) a size cap does not identify the

[4:51](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=291s) relevant messages. Selection flags on

[4:54](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=294s) the other hand let the agent tell your

[4:56](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=296s) tool which part it wants.

[4:59](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=299s) So based on my experience and

[5:01](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=301s) improvements I made to my own tool, I'll

[5:04](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=304s) give you three examples. Read the left

[5:06](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=306s) column as the bug reports. So starting

[5:09](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=309s) with the first one filtering with RG and

[5:11](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=311s) Z means let me pick the part I want. So

[5:14](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=314s) read my command. So my read command view

[5:16](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=316s) can now return just the part around a

[5:18](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=318s) match only the matching messages or the

[5:21](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=321s) last few with a size gap.

[5:23](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=323s) Second example would be Python over

[5:25](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=325s) JSON. So Python over JSON means give me

[5:28](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=328s) fewer fields. So search got a compact

[5:30](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=330s) mode one line per hit about six times

[5:33](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=333s) smaller than JSON with the same results.

[5:37](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=337s) Inspect command in chat history

[5:39](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=339s) summarizes a session. About the third of

[5:42](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=342s) inspect calls came from agent looping

[5:44](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=344s) over several sessions by hand. As you

[5:46](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=346s) can see with the for loop that means let

[5:48](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=348s) me ask several about several at once and

[5:51](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=351s) now it's one call. As you can see my

[5:54](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=354s) road map was in the commands around my

[5:57](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=357s) command and I had to learn from that.

[6:00](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=360s) Infraost the cloud cost tool wrote up

[6:03](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=363s) the same discovery in May. Their agents

[6:05](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=365s) were writing jQ and Python pipelines

[6:07](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=367s) over JSON dumps. So they moved the

[6:09](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=369s) filtering into the CLI and added a

[6:11](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=371s) compact format and a skill to teach it.

[6:13](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=373s) Plain claw with none of that got zero of

[6:16](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=376s) six hard questions right in their

[6:17](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=377s) testing and with it six of six and cost

[6:21](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=381s) across the benchmark also fell 41%. On

[6:24](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=384s) hacker news, the person who wrote that

[6:26](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=386s) post said that reading those agent logs

[6:28](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=388s) felt like watching usability tests and I

[6:31](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=391s) couldn't agree more.

[6:33](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=393s) This part might sound trivial and

[6:35](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=395s) obvious. Once it's in the CLI, teach

[6:37](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=397s) your agent it's there with the skill

[6:39](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=399s) file or the health flag, but the agents

[6:42](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=402s) take the teaching literally. I made some

[6:44](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=404s) changes that turned my deep flag into a

[6:47](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=407s) no op. I left the skill telling the

[6:49](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=409s) agents to use it to see what they do.

[6:51](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=411s) Would they find out or not? 95 searches

[6:54](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=414s) of out of 95 still passed it.

[6:58](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=418s) After that, two sessions loaded a

[6:59](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=419s) rewritten skill and that no longer told

[7:02](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=422s) them to use it. None of the searches

[7:04](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=424s) passed D flag and both tried new flags

[7:07](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=427s) on their first read. So your skill.md,

[7:10](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=430s) your help text, your error messages are

[7:12](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=432s) the interface. That also means stale

[7:14](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=434s) dogs are bugs because your agent will

[7:16](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=436s) keep following them.

[7:19](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=439s) Once the new flags reached agents,

[7:22](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=442s) nearly every search used compact output

[7:24](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=444s) and nearly every reduced the new flags.

[7:27](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=447s) Pipe breed fell from 82% to about a

[7:30](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=450s) quarter.

[7:36](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=456s) You can get real levels and test cases

[7:38](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=458s) for your tool easily. If you use it

[7:40](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=460s) every day, they are already in your

[7:41](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=461s) transcripts. The commands agent really

[7:43](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=463s) ran are the test cases and what agent

[7:45](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=465s) did immediately next is a rough answer

[7:47](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=467s) key. replay those commands through your

[7:49](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=469s) old build and your new one. If you

[7:52](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=472s) change a ranking, check that result uh

[7:55](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=475s) the agent went on to use still comes out

[7:57](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=477s) on the top or not. If you change an

[7:59](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=479s) output format, check that nothing went

[8:02](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=482s) missing. If you change a default, see

[8:04](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=484s) which commands now behave differently.

[8:06](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=486s) And when an agent reader a call with

[8:08](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=488s) different arguments, you have probably

[8:10](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=490s) found a failing case. The answer key is

[8:12](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=492s) only a rough guess at what the agent

[8:14](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=494s) wanted. keep a few cases back and use

[8:16](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=496s) those to score your fix.

[8:20](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=500s) So after analyzing 20,000 invocation, if

[8:24](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=504s) I were to start all over again, this is

[8:26](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=506s) what I would build first. 55% of the

[8:29](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=509s) measured pipelines used only head, tail,

[8:32](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=512s) or set downstream that points to demand

[8:34](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=514s) for smaller slices. So ship ranges and

[8:37](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=517s) limits before anything fancy. A flag

[8:41](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=521s) only helps if agents reach for it.

[8:42](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=522s) GitHub CLI's list command have a limit

[8:45](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=525s) flag. An agent used it 81% of the time.

[8:48](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=528s) The counter example would be cargo for

[8:51](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=531s) cargo test quite mode prints a character

[8:54](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=534s) per test instead of a line and still

[8:56](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=536s) reports failures. Agent passed it on

[8:58](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=538s) about one cargo run in 10 and pipped

[9:01](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=541s) instead. So teach the flag. So teach the

[9:03](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=543s) flag in your help test and your skill

[9:06](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=546s) and make big outputs print less by

[9:09](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=549s) default and say how to get more. for

[9:11](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=551s) example, and with something like showing

[9:13](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=553s) 20 of 300, add offset flag 20 for the

[9:17](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=557s) next wage.

[9:20](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=560s) One way to ship several of these at once

[9:22](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=562s) is an agent mode. Your CLI notices an

[9:24](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=564s) agent is calling it and prints

[9:25](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=565s) differently. Agents like cloud code set

[9:28](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=568s) an environment variable in their cell so

[9:30](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=570s) your tool can spot them. Doug DB shipped

[9:33](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=573s) one at a terminal. You get a nicely

[9:35](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=575s) drawn table, but an agent gets a plain

[9:37](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=577s) one that says how many rows were left

[9:39](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=579s) out and how to get them. A huge query

[9:42](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=582s) stops early and says so. And errors come

[9:44](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=584s) back as JSON because agents read them

[9:46](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=586s) better. Is the same list as we have been

[9:49](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=589s) just through and it cut the output model

[9:52](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=592s) red by 59%.

[9:54](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=594s) However, the tokens saved were nearly

[9:56](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=596s) about.5% of all the tokens the model

[9:59](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=599s) read. So cost and runtime didn't

[10:01](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=601s) improve. If you remember, infra cost

[10:04](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=604s) agent started getting answers right.

[10:06](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=606s) Doug Dvs were already there. So measure

[10:09](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=609s) turns and write answers, not just

[10:11](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=611s) tokens.

[10:14](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=614s) You might be thinking, I'll ask my agent

[10:16](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=616s) to do all of this. They are not that

[10:19](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=619s) good at this yet. At least not

[10:21](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=621s) oneshotting it. The design is where it

[10:24](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=624s) goes wrong. Ask an agent to build a tool

[10:26](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=626s) for agent. It probably pictures a person

[10:28](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=628s) and then bolts the agent on. It copies

[10:30](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=630s) most common patterns it has seen and

[10:32](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=632s) most of those are poor. A study of

[10:35](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=635s) official MCP servers found most just rep

[10:38](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=638s) an existing API. Another check tool

[10:41](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=641s) descriptions from real MCP servers and

[10:43](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=643s) flagged issues in 97% of them. Docs have

[10:47](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=647s) the same gap. Researchers at each Zurich

[10:50](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=650s) tested context file written by agents.

[10:53](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=653s) They didn't help tasks succeed cost more

[10:55](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=655s) to run and did worse than files written

[10:57](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=657s) by people. Agents are already good at

[11:00](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=660s) other half of this talk. Give one

[11:02](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=662s) transcript and a test it can run and it

[11:05](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=665s) can find what's wrong and improve the

[11:07](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=667s) tool. Sometimes past what people wrote

[11:09](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=669s) by hand and tropic reported that on its

[11:12](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=672s) own internal tools as well. So you

[11:14](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=674s) decide what the tool should be and let

[11:16](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=676s) the agent check it against what really

[11:18](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=678s) happened.

[11:20](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=680s) You have seen the evidence for each of

[11:22](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=682s) these. So, I won't read them. Take a

[11:24](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=684s) moment to absorb these or screenshot.

[11:33](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=693s) To find your own list, back up your

[11:35](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=695s) transcripts first because clot code can

[11:37](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=697s) delete inactive ones after 30 days.

[11:40](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=700s) Then, search for them for a tool's name

[11:43](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=703s) followed by a pipe and other similar

[11:45](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=705s) patterns. If you sip ship an MCP server,

[11:48](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=708s) look for agent calling your tool again

[11:50](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=710s) straight away with different arguments.

[11:54](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=714s) If you remember one thing from this

[11:56](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=716s) talk, if you are building a tool for

[11:58](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=718s) your agent, your agent transcript are

[12:01](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=721s) the product analytics you probably

[12:02](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=722s) aren't reading.

[12:06](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=726s) The workaround is the bug report. So go

[12:08](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=728s) read your transcripts and if you need

[12:10](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=730s) any help doing that, check out chat

[12:12](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=732s) history on cargo or GitHub. It can do

[12:14](https://www.youtube.com/watch?v=rS36lYuvXQ8&t=734s) this for you as well. Thank you.
