# What's new from Anthropic and what's next

Alex Albert, Anthropic

AI Engineer World's Fair 2024

[Talk page](https://ai.engineer/talks/EuC1GWhQdKE-whats-new-from-anthropic-whats-next) · [Video](https://www.youtube.com/watch?v=EuC1GWhQdKE)

## What's new from Anthropic and what's next

From electric factories to Artifacts, Alex Albert traces how better models and new interfaces can change the work a product makes possible.

Basic familiarity with LLM prompts, API calls, and web application development will help with the examples.

### A new motor in the same factory

[0:17](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=17s)

Picture a factory in 1882: a large coal-fired steam engine at one end, overhead transmission lines running across the building, and hundreds of machines moving to the same rhythm. Alex Albert opens with the arrival of commercial electric power and a puzzle: why didn’t a superior power source immediately produce a more productive factory? His answer lies in the machinery surrounding the engine.

![Black-and-white photograph of rows of factory machines beneath a dense network of overhead belts and shafts.](https://ai.engineer/api/talk-essay-media/bef8c15441efeca545511bec88af857d03846d545828fc91111c781d84b669ca)

Factory machinery connected by overhead belts and shafts.

If one station needed power, the entire steam engine had to run. The transmission system also dictated where machines could sit, so the layout served power distribution rather than the manufacturing process or its workers. Replacing the steam engine with an electric motor removed the nearby coal furnace and made lighting easier, but left those constraints intact.

The larger change came when factories were redesigned around electricity. Smaller, specialized tools made the arrangement more flexible; workers could bring tools to an item instead of carrying the item back to a fixed workstation. **The productivity opportunity was in redesigning the process around the new capability**, not merely changing its power source.

### The magic-star-icon phase

[2:35](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=155s)

An LLM added behind a star-shaped button can preserve a product’s existing assumptions just as faithfully as an electric motor preserved a steam-powered factory’s layout. Albert draws a second comparison with early mobile development: shrinking a website onto a phone preceded applications designed around an always-available camera and GPS. Snapchat and Uber illustrate the latter approach—products whose behavior depended on what the new platform could do.

There are practical reasons AI products remain in this phase. LLMs are nondeterministic, reliability is difficult, and prompts can require repeated optimization. Developers are learning to build with a component that behaves differently from conventional software. In Albert’s account, the text box is still the main interaction pattern that has stuck; finding useful alternatives requires both stronger models and experiments with the product surface itself.

### From benchmark scores to iterative coding

[4:33](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=273s)

The model enabling the next set of examples is [**Claude 3.5 Sonnet**](https://www.anthropic.com/news/claude-3-5-sonnet), released the Thursday before this 2024 talk. It was the first release in the Claude 3.5 family, and Albert describes this middle-sized model as outperforming the previous flagship, **Claude 3 Opus**. He points to MMLU, HumanEval, GPQA, and tool-use evaluations, then turns from academic tests to application behavior.

Albert highlights a 200K-token context window for retrieval-augmented generation and claims near-perfect recall across it, without specifying the recall test conditions. A large context gives an application room to supply retrieved material, but that claim alone does not establish end-to-end RAG accuracy. For coding, his observation is more behavioral: Sonnet seems to understand debugging problems better and gets caught in repeated, unproductive loops less often than earlier models.

Pull requests provide a more concrete way to test multistep work. They have a defined objective, usually require several changes, and let a model iteratively write code and test its progress. Anthropic’s internal evaluation involved fixing bugs or adding functionality in open-source codebases with coding tools.

Albert reports scores of 64% for Claude 3.5 Sonnet and 38% for Claude 3 Opus on Anthropic’s internal pull-request evaluation. These figures describe that evaluation, rather than a general success rate for arbitrary pull requests; the talk does not provide its sample size, attempt budget, or full scoring protocol.

### A table image becomes editable data

[6:00](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=360s)

Vision extends the same model beyond text and code. Albert reports improvements over Claude 3 Opus on essentially every vision benchmark Anthropic tested. His concrete example is table transcription: supply an image of a table and ask the model to reproduce it in Markdown. He says the result was almost perfect and that he checked the numbers himself.

The Workbench frame shows the uploaded table and transcription request on the left, with the resulting earnings-history table on the right. This turns visual source material into a representation that can be copied, edited, or incorporated into another document—the kind of practical use that made vision particularly compelling to Albert.

![Workbench interface with an uploaded table and the prompt “Transcribe this into a table” on the left, and an earnings-history table on the right.](https://ai.engineer/api/talk-essay-media/62dec5ff93de3ecf46e8a9bafefa42a7e2756908b52805bda9ac54a735e72111)

Anthropic Workbench transcribes an uploaded table into a tabular response.

### Capabilities need affordable access

[6:36](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=396s)

The launch offered three access paths: Anthropic’s API, AWS Bedrock, and Vertex AI. That choice let developers use Claude through the provider that fit their environment. The prices and availability here describe the launch-era offering, while the later model and steering announcements describe forecasts made during the talk.

Claude 3.5 Sonnet’s launch pricing was $3 per million input tokens and $15 per million output tokens. Albert describes it as five times cheaper than Claude 3 Opus. His application-level point is that intelligence, speed, and cost must work together: a model becomes easier to embed in a product when its useful capabilities are also economical to invoke.

### Artifacts give the work its own surface

[7:20](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=440s)

**Artifacts separate generated content from the conversation used to create it.** An essay, SVG, or React website can occupy its own work surface while the chat remains the place to request changes. This is the first product example that directly answers the factory analogy: the interface changes to accommodate what the model produces.

The experience combines Sonnet’s coding, reasoning, and vision abilities. A screenshot or Figma diagram supplies a visual target; Claude produces code and components that a developer can use. Albert demonstrates the combination by recreating the `claude.ai` chat layout in React from a single screenshot. The displayed result places the recreated interface beside the conversation that produced it.

Albert describes the idea as having hidden in plain sight for more than a year and a half. A tweet shown afterward frames LLM product development as early in its S-curve. The encouraging implication is that a useful interaction pattern can still emerge from capabilities developers have already been experimenting with.

### Projects make context a shared resource

[8:37](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=517s)

[Projects](https://www.anthropic.com/news/projects), released the Tuesday before the talk, address another part of the workflow: grounding outputs in knowledge a team already has. Style guides, codebases, transcripts, and past work become context for collaboration with Claude. On the Claude Team plan, teammates can share projects and chats.

Albert reports that Anthropic engineers upload repositories and documentation, and that he has begun seeing people share chats and Artifacts in place of some Google Docs or site documentation. The Project view shows a chat input alongside a populated Project knowledge sidebar. The shared object is now more than a finished document: it includes the knowledge and conversation involved in creating the work.

![Claude Project page with a central chat input and a Project knowledge sidebar listing uploaded source files.](https://ai.engineer/api/talk-essay-media/7ebe867da7144585a9001a0e51623fb4d46f8d61ee529ae8317aed36f88bf6f2)

A Claude Project with source files listed in Project knowledge.

### Tools connect model output to application behavior

[9:29](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=569s)

The developer-facing counterpart is the [Tool Use API](https://claude.com/blog/tool-use-ga), which Albert says had launched about a month earlier. It lets an application supply custom client-side functions for Claude to select. Tool use also provides a way to request consistent structured JSON, so the model’s response can fit an application’s data contract rather than arrive only as prose.

For example, a tool definition for extracting a task from text can describe the expected fields explicitly:

```json
{
  "name": "record_task",
  "description": "Extract a task and its owner from the supplied text.",
  "input_schema": {
    "type": "object",
    "properties": {
      "task": {
        "type": "string",
        "description": "The work to be completed."
      },
      "owner": {
        "type": "string",
        "description": "The named owner, or an empty string if none is given."
      }
    },
    "required": ["task", "owner"]
  }
}
```

The schema makes the desired structure explicit; a tool request still needs application code to handle it. Albert also reports seeing developers give Sonnet hundreds of tools at once, an observation about emerging usage rather than a measured reliability result.

### A shorter path from task description to prompt testing

[10:08](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=608s)

The developer console reduces the distance between describing a task and trying a prompt:

1. Enter a task description into the prompt generator. Claude produces a prompt intended to suit that task.
2. Open the generated prompt in the Workbench and edit it directly.
3. Use variables to turn the prompt into a template, substituting inputs for cases such as RAG testing.

This gives developers a starting point and a place to iterate, rather than requiring every prompt to begin as a blank text field.

Albert then previews the console’s Evaluate feature, already visible with a beta tag. He promises further details and iteration; the demonstration at this point establishes the direction toward evaluation without walking through a complete testing or grading workflow.

### Plan around a moving capability frontier

[10:52](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=652s)

The roadmap preview names **Claude 3.5 Haiku** and **Claude 3.5 Opus** as models expected later that year. Albert describes the goals for each generation as greater intelligence, lower latency, and lower cost. His advice to developers follows from that pace: plan ambitiously enough that a new model arriving during development can expand what the product does. He expects meaningful improvement on a timescale of months rather than years.

### Controlling features as well as prompts

[11:31](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=691s)

A separate research direction asks whether developers can influence model behavior by manipulating internal features. Albert introduces *Scaling Monosemanticity*, covered in Anthropic’s [research account](https://www.anthropic.com/news/mapping-mind-language-model). The research concerns **Claude 3 Sonnet**, distinct from the newly released 3.5 model. It identifies features that activate for different topics; clamping a feature’s value higher or lower can steer the output.

[Golden Gate Claude](https://www.anthropic.com/news/golden-gate-claude) made that mechanism visible to users by amplifying the Golden Gate Bridge feature. Instead of relying only on instructions in a prompt, the demonstration altered an internal feature’s influence. Albert then describes a few beta testers experimenting with a steering API that lets them find and clamp features for specific attributes. Broader developer access is something he hopes to offer, not a general release announced here.

### Build a prototype and start the feedback loop

[12:28](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=748s)

The immediate next step is a prototype: make something quickly enough to begin validating it and learning from feedback. To encourage that work, Albert announces a Build with Claude contest launched the previous day. Albert says the contest runs until July 10, with each of the top three projects receiving $10,000 in Anthropic API credits. He directs attendees to the on-screen destination and the top of the documentation page for the contest details.

For questions beyond the session, Albert offers to talk at the AWS booth down the hall over the next few hours. He also invites questions on X/Twitter and says he tries to read all his direct messages. The invitation leaves the next product experiment with the developers in the room.

## Resources

- [Claude 3.5 Sonnet and Artifacts launch](https://www.anthropic.com/news/claude-3-5-sonnet) — Anthropic's historical announcement covering Sonnet's evaluations, launch pricing, availability and Artifacts preview.
- [Collaborate with Claude on Projects](https://www.anthropic.com/news/projects) — How Projects introduced reusable knowledge, custom instructions and shared team conversations.
- [Claude can now use tools](https://claude.com/blog/tool-use-ga) — The historical tool-use announcement explains external-tool integration and structured data extraction.
- [Mapping the mind of a large language model](https://www.anthropic.com/news/mapping-mind-language-model) — An accessible introduction to the feature-extraction and steering research behind Scaling Monosemanticity.
- [Golden Gate Claude](https://www.anthropic.com/news/golden-gate-claude) — Anthropic's account of the demonstration that amplified a Golden Gate Bridge feature.
- [Evaluate prompts in the developer console](https://claude.com/blog/evaluate-prompts) — The July 2024 announcement explains test suites, generated inputs, output comparisons and human grading in the Console.

## Slides

### Presenting sponsor

Microsoft is the presenting sponsor.

[0:05](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=5s) [Microsoft presenting sponsor slate on a black background.](https://assets.ai.engineer/sites/aiecode2025/sha256/053f2dac4c10b9b6e50155cb74972fe7e0a7caee31b574c53a03c853a131eca7.webp)

PRESENTING SPONSOR

Microsoft

### Factory with overhead drive belts

Overhead belts linked the factory’s machines to a single steam engine. Powering one station meant running the whole system, and the transmission lines constrained the layout.

[1:17](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=77s) [Historic photograph of factory machinery driven by overhead belts.](https://assets.ai.engineer/sites/aiecode2025/sha256/82ffbd7ab8e6dba0d99190711816550a22b6f5d88e1e026b9762d29337e76190.webp)

### Electrical machinery

Many factory owners initially replaced steam engines with electric ones, leaving fundamental limitations intact. The speaker argues that the real shift came from redesigning factories around electricity.

[1:48](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=108s) [Black-and-white illustration of an early electrical machine.](https://assets.ai.engineer/sites/aiecode2025/sha256/4ad62439bda4f8c3522c63d319a3c26267e454a8499a01958e632d82a6f7e0ad.webp)

### Workers in a factory

Electricity let factories reorganize around smaller, specialized tools: workers could bring tools to the work instead of moving items to workstations. The speaker draws a parallel to how products add AI today.

[2:17](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=137s) [Historic photograph of workers assembling machinery in a factory.](https://assets.ai.engineer/sites/aiecode2025/sha256/a60eb4c2cd2f92b6f3f93a1395f7bdbdb629c8db4a3cc7397aada90ee95bda1d.webp)

987

### The AI feature meme

The meme jokes about an AI magic-wand button that loads for 10 seconds and a separate sparkle button. The speaker says developers are still early: LLMs are nondeterministic, reliability is an issue, and prompts need repeated optimization.

[3:44](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=224s) [A four-panel cartoon meme jokes about adding slow magic-wand and sparkle buttons to an app.](https://assets.ai.engineer/sites/aiecode2025/sha256/2e5ee0f15d7f6b560063d2faf7b7a4c2c8b7d4c4e071326c21c563e70e88c9a5.webp)

🪄✨

GUESS WHAT NEW FEATURE
I ADDED TO YOUR APP

IT'S AI!

WATCH AS THE ~MAGIC WAND~
BUTTON LOADS FOR 10 SECONDS

BUT WAIT!
THERE'S ALSO A ✨ BUTTON!

BOOK OF SPEL[illegible]

Interface text:
near
@nearcyan
Subscribe

### Claude text box

The speaker says few AI product ideas have stuck beyond the text box, leaving more opportunities to explore.

[4:12](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=252s) [A large text box asks how Claude can help today.](https://assets.ai.engineer/sites/aiecode2025/sha256/f6bb8ad5ba2033e5753d60e27a52968237332cdaed6d980d60b44f620f41c3d3.webp)

How can Claude help you today?

### Claude 3.5 Sonnet

Anthropic introduces Claude 3.5 Sonnet, the first model in the Claude 3.5 family. The speaker says this middle model outperforms the previous best model, Claude 3 Opus.

[4:32](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=272s) [A presentation screen displays the title Claude 3.5 Sonnet beside the speaker on stage.](https://assets.ai.engineer/sites/aiecode2025/sha256/275ffc5db340f70207056a8b17f9bc8e8d95cf47a1d0d431903826311ac54b4d.webp)

Claude 3.5 Sonnet

### Claude model intelligence and cost

The chart places Claude 3.5 Sonnet above Claude 3 Sonnet at the same cost, and above Claude 3 Opus at a lower cost.

[4:38](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=278s) [A chart places Claude 3.5 Sonnet above Claude 3 Sonnet in intelligence at the same cost, alongside Claude 3 Haiku and Opus.](https://assets.ai.engineer/sites/aiecode2025/sha256/2c8a00092a14330605f80991831ba4fc71d3a6f254d2de4af21c9e40f5a1fdc1.webp)

INTELLIGENCE
Benchmark scores

COST
Price per million tokens

Claude 3 Haiku
Claude 3 Sonnet
Claude 3.5 Sonnet
Claude 3 Opus

### Claude 3.5 Sonnet benchmark comparisons

Claude 3.5 Sonnet leads the highlighted ranking with 90 overall accuracy. On GPQA Diamond, its 0-shot CoT accuracy distribution sits above GPT-4o’s across 20 runs.

[5:02](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=302s) [Overlapping benchmark tables and a GPQA accuracy chart compare Claude 3.5 Sonnet with other models.](https://assets.ai.engineer/sites/aiecode2025/sha256/768fd78b753fcf12308b17a4979a12ff7bf92872743a0cc9bdbf206f647edf7c.webp)

Claude 3.5 Sonnet | Claude 3 Opus | GPT-4o | Gemini 1.5 Pro | Llama-400b (early snapshot)

Graduate level reasoning — GPQA, Diamond
Claude 3.5 Sonnet: 59.4%* — 0-shot CoT
Claude 3 Opus: 50.4% — 0-shot CoT
GPT-4o: 53.6% — 0-shot CoT
Gemini 1.5 Pro: —

Undergraduate level knowledge — MMLU
Claude 3.5 Sonnet: 88.7%** — 5-shot; 88.3% — 0-shot CoT
Claude 3 Opus: 86.8% — 5-shot; 85.7% — 0-shot CoT
GPT-4o: —; 88.7% — 0-shot CoT
Gemini 1.5 Pro: 85.9% — 5-shot; —

Code — HumanEval
Claude 3.5 Sonnet: 92.0% — 0-shot
Claude 3 Opus: 84.9% — 0-shot
GPT-4o: 90.2% — 0-shot
Gemini 1.5 Pro: 84.1% — 0-shot

Multilingual math — MGSM
Claude 3.5 Sonnet: 91.6% — 0-shot CoT
Claude 3 Opus: 90.7% — 0-shot CoT
GPT-4o: 90.5% — 0-shot CoT
Gemini 1.5 Pro: 87.5% — 8-shot

Reasoning over text — DROP, F1 score
Claude 3.5 Sonnet: 87.1 — 3-shot
Claude 3 Opus: 83.1 — 3-shot
GPT-4o: 83.4 — 3-shot
Gemini 1.5 Pro: 74.9 — Variable shots

Mixed evaluations — BIG-Bench-Hard
Claude 3.5 Sonnet: 93.1% — 3-shot CoT
Claude 3 Opus: 86.8% — 3-shot CoT
GPT-4o: —
Gemini 1.5 Pro: 89.2% — 3-shot CoT

Math problem-solving — MATH
Claude 3.5 Sonnet: 71.1% — 0-shot CoT
Claude 3 Opus: 60.1% — 0-shot CoT

Grade school math — GSM8K
Claude 3.5 Sonnet: 96.4% — 0-shot CoT
Claude 3 Opus: 95.0% — 0-shot CoT

* Claude 3.5 Sonnet scores 67.2% [illegible]
** Claude 3.5 Sonnet scores 90.4% [illegible]

Rank | Overall Acc | Model | Cost ($) | Average Latency (s) | AST Summary | Exec Summary | Relevance
1 | 90 | Claude-3.5-Sonnet-20240620 (Prompt) | 2.2 | 1.37 | 91.31 | 89.5 | 85.42
2 | 88 | GPT-4-0125-Preview (Prompt) | 5.25 | 1.97 | 91.22 | 88.1 | 70.42
3 | 87.65 | Claude-3-Opus-20240229 (Prompt) | 10.84 | 4.61 | 88.93 | 86.16 | 80.42
4 | 86.35 | Gemini-1.5-Pro-Preview-0514 (FC) | 0.86 | 1.94 | 87.92 | 83.32 | 89.58
5 | 85.88 | Gemini-1.5-Pro-Preview-0409 (FC) | 0.86 | 1.95 | 87.62 | 82.88 | 88.75
6 | 85.88 | GPT-4-1106-Preview (FC) | 5.07 | 5.97 | 88.62 | 81.73 | 80.42

GPQA Accuracies
Models: GPT-4o; Claude 3.5 Sonnet
X-axis: GPQA Accuracy (Diamond, 0-shot CoT, 20 runs)
X-axis ticks: 35%, 40%, 45%, 50%, 55%, 60%, 65%

### Claude 3.5 Sonnet context recall

The heatmap stays near the top of its 0–1 scale across needle positions and context lengths up to 200k. The speaker describes this as near-perfect recall across the context.

[5:12](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=312s) [Heatmap of needle position versus context length, with nearly all cells dark green.](https://assets.ai.engineer/sites/aiecode2025/sha256/20cb406fc183de27f2b8443109ab0ae3f9a5ada31b01dfcff39780bd29cd8901.webp)

Claude 3.5 Sonnet

Y-axis: Needle position (%)
Y-axis ticks: 6, 12, 19, 25, 31, 38, 44, 50, 56, 62, 69, 75, 81, 88, 94

X-axis: Context length
X-axis ticks: 12k, 25k, 38k, 50k, 62k, 75k, 88k, 100k, 112k, 125k, 138k, 150k, 162k, 175k, 188k, 200k

Color scale: 0, 1

### Internal pull request evaluation pass rates

Claude 3.5 Sonnet passed 64% of internal pull request evaluations, compared with 38% for Claude 3 Opus. The tasks measure multi-step reasoning as the model iteratively writes and tests a solution.

[5:21](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=321s) [Bar chart comparing pull request evaluation pass rates across four Claude models, with Claude 3.5 Sonnet highest.](https://assets.ai.engineer/sites/aiecode2025/sha256/58202a51888727330904daeac939334c44e5f497faf8bd3d1d28b991979eac97.webp)

Internal Pull Request Eval Pass Rates

Y-axis: Pass Rate (%)
Y-axis ticks: 0, 20, 40, 60, 80
Models: Claude 3.5 Sonnet, Claude 3 Opus, Claude 3 Sonnet, Claude 3 Haiku

### Visual benchmark comparisons

Claude 3.5 Sonnet scores above Claude 3 Opus on all five visual benchmarks and leads the four-model comparison on four. GPT-4o leads on visual question answering.

[5:58](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=358s) [Table compares four models on visual math, science diagrams, visual questions, charts and documents.](https://assets.ai.engineer/sites/aiecode2025/sha256/3cfd17d5741418ba59f582c2aaa7f1fb11fe7a37b17d6007bfcec14bd2db6763.webp)

Models: Claude 3.5 Sonnet | Claude 3 Opus | GPT-4o | Gemini 1.5 Pro

Visual math reasoning
MathVista (testmini)
67.7% (0-shot CoT) | 50.5% (0-shot CoT) | 63.8% (0-shot CoT) | 63.9% (0-shot CoT)

Science diagrams
AI2D, test
94.7% (0-shot) | 88.1% (0-shot) | 94.2% (0-shot) | 94.4% (0-shot)

Visual question answering
MMMU (val)
68.3% (0-shot CoT) | 59.4% (0-shot CoT) | 69.1% (0-shot CoT) | 62.2% (0-shot CoT)

Chart Q&A
Relaxed accuracy (test)
90.8% (0-shot CoT) | 80.8% (0-shot CoT) | 85.7% (0-shot CoT) | 87.2% (0-shot CoT)

Document visual Q&A
ANLS score, test
95.2% (0-shot) | 89.3% (0-shot) | 92.8% (0-shot) | 93.1% (0-shot)

### AMZN earnings table transcription

Claude 3.5 Sonnet transcribes an image of Amazon’s earnings history into a Markdown table, preserving the dates, forecasts, EPS figures and year-over-year changes.

[6:09](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=369s) [Earnings history table lists report dates, fiscal quarters, forecasts, earnings per share and year-over-year changes.](https://assets.ai.engineer/sites/aiecode2025/sha256/f599a1e1089960d969a77727936b190b56eeaca5fa30389fefc503870d1f0466.webp)

AMZN Earnings History

Report Date | Fiscal Quarter | Forecast / EPS | Last Year's EPS | EPS YoY Change
Jul 25, 2024 | 2024 (Q2) | 1.03 / - | 0.65 | —
Apr 30, 2024 | 2024 (Q1) | 0.84 / 0.98 | 0.31 | 216.13% (+0.67)
Feb 01, 2024 | 2023 (Q4) | 0.80 / 1.00 | 0.03 | 3233.33% (+0.97)
Oct 26, 2023 | 2023 (Q3) | 0.58 / 0.94 | 0.28 | 235.71% (+0.66)
Aug 03, 2023 | 2023 (Q2) | 0.35 / 0.65 | -0.2 | 425.00% (+0.85)
Apr 27, 2023 | 2023 (Q1) | 0.21 / 0.31 | -0.378 | 182.01% (+0.69)
Feb 02, 2023 | 2022 (Q4) | 0.17 / 0.03 | 1.388 | -97.84% (-1.36)
Oct 27, 2022 | 2022 (Q3) | 0.22 / 0.28 | 0.306 | -8.50% (-0.03)
Jul 28, 2022 | 2022 (Q2) | 0.12 / -0.20 | 0.756 | -126.46% (-0.96)
Apr 28, 2022 | 2022 (Q1) | 0.42 / -0.38 | 0.79 | -147.85% (-1.17)

[6:14](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=374s) [Workbench displays an earnings table transcribed from an image.](https://assets.ai.engineer/sites/aiecode2025/sha256/1483d325961ba85869dbe273ea31c4d8910d81898309c491c3289c984e6468a4.webp)

USER
Transcribe this into a table

RESPONSE
Here's the transcription of the AMZN Earnings History into a table format:

Report Date | Fiscal Quarter | Forecast / EPS | Last Year's EPS | EPS YoY Change
Jul 25, 2024 | 2024 (Q2) | 1.03 / - | 0.65 | —
Apr 30, 2024 | 2024 (Q1) | 0.84 / 0.98 | 0.31 | 216.13% (+0.67)
Feb 01, 2024 | 2023 (Q4) | 0.80 / 1.00 | 0.03 | 3233.33% (+0.97)
Oct 26, 2023 | 2023 (Q3) | 0.58 / 0.94 | 0.28 | 235.71% (+0.66)
Aug 03, 2023 | 2023 (Q2) | 0.35 / 0.65 | -0.2 | 425.00% (+0.85)
Apr 27, 2023 | 2023 (Q1) | 0.21 / 0.31 | -0.378 | 182.01% (+0.69)
Feb 02, 2023 | 2022 (Q4) | 0.17 / 0.03 | 1.388 | -97.84% (-1.36)
Oct 27, 2022 | 2022 (Q3) | 0.22 / 0.28 | 0.306 | -8.50% (-0.03)
Jul 28, 2022 | 2022 (Q2) | 0.12 / -0.20 | 0.756 | -126.46% (-0.96)
Apr 28, 2022 | 2022 (Q1) | 0.42 / -0.38 | 0.79 | -147.85% (-1.17)

Interface text:
ANTHROPIC
Dashboard
Workbench
Settings
Docs
AA
Untitled - 2024-06-24 12:30:11 AM
v1
Prompt
Evaluate BETA
Get Code
Run
SYSTEM PROMPT
Set a system prompt (optional)
Markdown
Text
Add messages

### Model availability

Claude 3.5 Sonnet is available through Anthropic’s API, Amazon Bedrock, and Vertex AI, giving developers a choice of where to use it.

[6:48](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=408s) [Anthropic, Amazon Bedrock, and Vertex AI logos.](https://assets.ai.engineer/sites/aiecode2025/sha256/c41bafb58bb1bb5c080e953aee567deed1926cb4c8b423145c5ad1215a25d42a.webp)

vertex.ai

Amazon Bedrock

### Claude 3.5 Sonnet pricing

Anthropic priced Claude 3.5 Sonnet at $3 per million input tokens and $15 per million output tokens, five times cheaper than Claude 3 Opus.

[6:59](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=419s) [Pricing shown as $3 per million input tokens and $15 per million output tokens.](https://assets.ai.engineer/sites/aiecode2025/sha256/cec5d816af52164cdf437c23a6e4c0d374a3f07a03b1e75d5082a7915790202b.webp)

$3 per million input tokens
$15 per million output tokens

### Artifacts

Artifacts separate Claude’s generated content from the chat, letting users collaborate with Claude on essays, SVGs, and React websites.

[7:17](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=437s) [A presenter stands beside a projected slide titled Artifacts.](https://assets.ai.engineer/sites/aiecode2025/sha256/645a5275e8c207eddfe674d276d0257f757276b602d914f97b33e57a4c300a3e.webp)

Artifacts

Event signage: AI Engineer World's Fair; Presented by Microsoft. Sponsor logos include Matillion, Vespa, Snyk, Writer, Databricks, FriendliAI, Cleric, Elastic, Move AI, Airtrain.ai, Trunk Tools, Substrate, Deepgram, Weights & Biases, Microsoft, AWS, Neo4j, Google Cloud, Sourcegraph, MongoDB, DataStax, Galileo, Neptune.ai, Covalent, Hasura, Crusoe, Groq, and Emergence.

### Image editing app artifact

Claude responds to a cropping bug while the artifact’s code appears in a separate panel. Its proposed fix draws the cropped image onto a temporary canvas before creating a new image object.

[7:33](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=453s) [Claude discusses an image cropping fix beside an artifact's code panel.](https://assets.ai.engineer/sites/aiecode2025/sha256/3c513bbc73b07385ffe950b0c01a15fef4078a2c754fda7ca3d87ce887c58cac.webp)

After I crop, the image turns white

I apologize for this issue. It seems there's a problem with how we're handling the cropped image data. Let's investigate and fix this issue so that the cropped image displays correctly. I'll update the Dingboard app to resolve this problem.

Dingboard - Image Editing App (Fixed Crop Functionality)
Click to open website

I've updated the Dingboard app to fix the issue where the cropped image was turning white. Here are the key changes and explanations:

1. Improved Cropping Process:
• Instead of using toDataURL() on the entire canvas, we now create a temporary canvas to perform the crop.
• We draw only the cropped portion of the image onto this temporary canvas.
• We then create a new fabric.Image object from the cropped canvas data.

Code panel:
        top: 100,
        fontFamily: 'Arial',
        fill: '#000000',
        fontSize: 20
    });
    canvas.add(text);
    canvas.setActiveObject(text);
    bringTextToFront();
    canvas.renderAll();
});

// Function to bring all text objects to front
function bringTextToFront() {
    canvas.getObjects().forEach(obj => {
        if (obj.type === 'i-text') {
            canvas.bringToFront(obj);
        }
    });
}

// Initialize select mode
setMode('select');
</script>
</body>
</html>

Interface text:
Claude
Dingboard: React-based Image Editing App
Share
Dingboard - Image Editing App
Preview
Code
Reply to Claude...
Claude 3.5 Sonnet
2
Version 8 of 8

### Recreating an interface from a screenshot

Alex Albert shows Claude turning a screenshot of its chat interface into a React component, with the generated interface displayed beside the conversation.

[7:46](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=466s) [A social post shows Claude recreating a chat interface from a screenshot.](https://assets.ai.engineer/sites/aiecode2025/sha256/9a6a740bd8384493d2470fddfdd4065ff6b6bc7a03a436bbfbda1569fe7daf2e.webp)

Alex Albert
@alexalbert__

Claude can remake the claude dot ai UI from just a single screenshot.

Can you remake this website with React in an artifact?

Anthropic Chat Interface React Component
Click to open component

I've created a React component that replicates the main elements of the Anthropic chat interface shown in the image. Here's a breakdown of the key features:
1. The component uses Tailwind CSS for styling, as required.
2. It includes the header with the Anthropic logo, "Untitled" dropdown, and icons.
3. The main content area displays the "What can I help you with today?" message with the custom star-like icon.
4. The footer contains an input field for messaging Claude, along with the model name and token count.
5. I've used icons from the Lucide React library where appropriate.

What can I help you with today?

Interface text:
React Component for Anthropic Chat Interface
Anthropic Chat Interface React Component
Preview
Code
Untitled
Message Claude...
Claude 3.5 Sonnet
2
Reply to Claude...
Claude 3.5 Sonnet
2
Version 4 of 4

### Artifacts and the assistant product curve

Fraser sees Artifacts as evidence that assistant products are still early on the S curve. The speaker finds that possibility inspiring.

[8:16](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=496s) [A post suggests that Artifacts shows assistant products are early in their development curve.](https://assets.ai.engineer/sites/aiecode2025/sha256/6d0d7ff1991885d792891b9f2eff73b79984c2dd65c5783bb139f499d09b85d5.webp)

fraser
@Fraser

Perhaps the most inspiring part of the Claude 3.5 release is that Artifacts is evidence we are early in the S Curve for productizing an assistant

### Adding project knowledge

Three PDFs are dragged toward project knowledge, where Claude can use customer feedback, survey results, and an NPS score to inform its responses.

[8:42](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=522s) [Files are being added to a Claude project's knowledge area.](https://assets.ai.engineer/sites/aiecode2025/sha256/aa089ddf8dfa40792891114ba797f9953cd96ca8425a7b61ce14f24e90db75ce.webp)

[illegible]er Call Transcripts

[illegible]ntains the call transcripts of a [illegible]stomer, Acme Inc.

Project knowledge
Add

Add to project knowledge
Claude will use the project knowledge uploaded to respond to you.

Customer feedback transcript.pdf
Survey results.pdf
NPS score.pdf

Interface text:
Customer Call Transcript...
Ahsam
3

### A project with code in its knowledge area

A Claude project lists Swift files in its knowledge area for Android work. The speaker says Anthropic engineers upload code repositories and documentation, then share chats and artifacts with teammates.

[8:54](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=534s) [A Claude project page shows a list of code files in its project knowledge panel.](https://assets.ai.engineer/sites/aiecode2025/sha256/a6643507dff7a23f79ebfab8b1ac1724c8ca7fa8b2ecd92bc6fa27000c095893.webp)

iOS & Androidw
Having Claude write android for me
Created by Mike Lew

Project knowledge
Project is Open. Everything added to project knowledge will be viewable by your org.
92% of knowledge size used

WithoutAnimations — SWIFT — Last month
ViewExtensions — SWIFT — Last month
Uuid — SWIFT — Last month
ToastScreen — SWIFT — Last month
ToastErrorBox — SWIFT — Last month
ThemeFonts — SWIFT — Last month
ThemeColors — SWIFT — Last month
TextSelectionViewController — SWIFT — Last month
Segment — SWIFT — Last month
ScreenProvider — SWIFT — Last month
ScreenAnalyticsLogging — SWIFT — Last month

Interface text:
Claude
All projects
Add Content
How can Claude help you today?
Claude 3.5 Sonnet
1
Add content
iOS & Androidw
Activity
Your chats
Your chats are private until shared
AA

### Tool use

Tool Use gives Claude custom client-side functions and enables consistent structured JSON output. The speaker has seen developers give Claude 3.5 Sonnet hundreds of tools at once.

[9:58](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=598s) [A hand points at illustrated controls beneath the title Tool use.](https://assets.ai.engineer/sites/aiecode2025/sha256/ed3b187b92227afc103973191ef4b134b402e8378950875da2de04be9d440563.webp)

Tool use

### Prompting assistance for content moderation

The demo introduces a Console prompt generator that uses Claude to turn a task description into an optimized prompt, with content moderation as the example.

[10:07](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=607s) [Video title card introducing prompting assistance in the Console for content moderation.](https://assets.ai.engineer/sites/aiecode2025/sha256/52eca76c673214fe89d19658c7437d5f0eabc88ce5d7b54499b451a45f7ad75e.webp)

Get prompting assistance,
directly in Console

Content moderation

Interface text:
Prompting assistance in Console: Content moderation
Watch later
Share
0:02 / 0:45
CC
HD
YouTube

### Generated moderation prompt

The prompt generator turns a moderation task description into a prompt with policy and transcript variables. The prompt can then be edited in the workbench.

[10:17](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=617s) [Generated prompt defining a content moderation task with policy and transcript placeholders.](https://assets.ai.engineer/sites/aiecode2025/sha256/e33654468a05d32399a8bab7e3ce0c1932cb6254012182f5d619c5b23c648a76.webp)

Your generated prompt

Click Start Editing to use this prompt, or discard it to generate a new one.

You will be acting as a content moderator to classify chat transcripts as either approved or rejected based on a provided content moderation policy. Here is the policy you should enforce:

<content_policy>
{{CONTENT_POLICY}}
</content_policy>

Here is the chat transcript to review and classify:

<transcript>
{{TRANSCRIPT}}
</transcript>

Carefully review the content policy. Then review the

Interface text:
Cancel
Start Editing

## Full transcript

[0:00](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=0s) [upbeat music] Morning, everyone. Good morning.

[0:17](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=17s) So today, I want to start with a, uh, a little story. Um, a short history lesson, if you will. So, you know, sit back, get comfortable. Uh, I'm gonna take us back to the year 1882.

[0:31](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=31s) It's the dawn of the electrical revolution, really. The world's first commercial power plant just opened up.

[0:38](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=38s) Electricity, this amazing new force, is all the rage in the manufacturing industry.

[0:44](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=44s) People are claiming that it's gonna change everything.

[0:47](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=47s) And yet, something very interesting happened around this time,

[0:52](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=52s) or rather, it didn't happen. You see, despite electricity's obvious superiority in comparison to the traditional techniques at the time, like steam engines, it didn't immediately improve manufacturing productivity.

[1:06](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=66s) Why? Well, because factory owners were simply trying to replace their old technology with this new technology into an outdated paradigm.

[1:17](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=77s) Let's picture a typical factory at the time. So we have a huge coal-fired steam engine on one end, and we have a network of transmission lines running across the top, all driving hundreds of machines locked in the same rhythm.

[1:29](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=89s) These legacy steam-powered factories were incredibly inefficient. You know, if one station needed power, all of a sudden you had to turn on that entire steam engine and it had to power all of them.

[1:39](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=99s) Factory layouts were dictated by the limitations of the transmission lines, not by what was best for the process or for the workers.

[1:48](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=108s) When electricity arrived, many factory owners simply swapped out the steam engine for an electric one. And sure, you know, they added some lights and, you know, workers didn't have to toil next to a coal-fired furnace all day, but the fundamental limitations of the factory remained.

[2:05](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=125s) So this real electrical revolution, well, it didn't actually come until we imagined factories from the ground up with electricity at its core.

[2:16](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=136s) Factories started to become flexible and adaptable. They allowed for smaller, specialized tools. Workers could bring their tools to the items instead of having to lug the items back to their workstations.

[2:29](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=149s) The entire manufacturing process became more efficient, more humane, and more productive.

[2:35](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=155s) Now let's fast-forward 140-something years to today, and you can see that we find ourselves at a similar point in regards to AI and LLMs. Enterprises, startups, developers are all building and integrating LLMs into their products, but often they're just tacking it onto their existing product surface, adding a few star icon buttons in the top left corner and

[2:56](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=176s) calling it a day. And this is not the first time we've seen this in Silicon Valley. [clears throat] Let's think back to when mobile first emerged, right? Companies simply tried to just shrink down their website and put it on a phone.

[3:11](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=191s) It wasn't until we redesigned apps from the ground up with the unique capabilities of mobile, like always-on camera and GPS, that we actually began to see true innovation in the space and adoption.

[3:24](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=204s) This is when the Snapchats and the Ubers of the world started to emerge.

[3:29](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=209s) So just as companies-- And s- just as factories went through their, you know, replace steam engines with electric one phase, and tech companies went through their just hire a couple mobile web dev people phase, we're now in our magic star icon phase with respect to AI.

[3:46](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=226s) And, uh, yeah, it's funny, but the thing is you can't blame any of the companies or developers that are actually trying to do this right now, right? Like, all of us are trying to do this, but in many ways we're just still so early.

[3:58](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=238s) LLMs are non-deterministic. They're hard to build on. They're completely different than what most developers are used to using.

[4:05](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=245s) Reliability is still an issue. Prompts still take rounds and rounds of optimization.

[4:12](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=252s) And we've also just started to scratch the surface of potential product opportunities. So far, not much has really stuck beyond just the text box.

[4:21](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=261s) We've been missing something, something that's a little hard to put a finger on. But just last week, I think we scratched the surface of a potential new product feature that we can build.

[4:33](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=273s) As some of you may have heard, last Thursday we released our new model, Claude 3.5 Sonnet.

[4:39](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=279s) 3.5 Sonnet is the first model that we released in the new Claude 3.5 family. It's only the middle model, and yet it is better than our last best model, Claude 3 Opus.

[4:51](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=291s) In my opinion, Claude 3.5 Sonnet is one of the best models in the world right now, and the benchmarks seem to back it up.

[4:59](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=299s) MMLU, HumanEval, GPQA, tool use, all the common characters here. It's top of its class in many regards in these academic lab-type environments, but what I am most excited for is how it actually does in the real world.

[5:14](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=314s) The model is particularly strong in RAG use cases, thanks to its 200K context. It also has near-perfect recall over that entire context as well. On coding tasks, 3.5 Sonnet seems to grasp debugging problems better.

[5:28](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=328s) It's not getting stuck in those same loops as much as previous models.

[5:33](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=333s) One of the best methods that we've found for actually measuring more complicated chains of reasoning is pull requests. They have a defined task, they usually take a few steps to solve, and the model's able to iteratively write and test its way to a solution.

[5:48](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=348s) In our own internal pull request evals, we're seeing that Claude 3.5 Sonnet scores a sixty-four percent. And to put that number in comparison, Claude 3 Opus only scored a thirty-eight percent.

[6:00](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=360s) 3.5 Sonnet also has state-of-the-art vision abilities. It shows considerable improvement over 3 Opus in basically every benchmark that we tested it on.

[6:10](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=370s) Things like table transcriptions and OCR are a breeze now. Passed this table in a s- uh, 3.5 Sonnet and basically replicated it perfectly in markdown. Um, probably can't read all those numbers, but trust me, I, I double-checked them to make sure they're all right.

[6:25](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=385s) Uh, vision capabilities were actually what amazed me the most when I started playing around with this model. It feels like we are really on the cutting edge of unlocking so many more use cases.

[6:36](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=396s) And, you know, as you're hearing me say all this, you might be thinking, "Well, that's great, Alex, but, I mean, it doesn't mean anything if I can't actually use the model."

[6:42](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=402s) And you're right, and we heard you, and that's why 3.5 Sonnet is available on our API, AWS Bedrock, and Vertex AI.

[6:51](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=411s) We understand that developers want choice when they're building, and we want Claude to be available wherever you are.

[6:59](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=419s) In terms of pricing, 3.5 Sonnet is five times cheaper than 3 Opus. It's only $3 per million input tokens and $15 per million output tokens.

[7:10](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=430s) 3.5 Sonnet's combo of speed, intelligence, and low cost makes it much more economical to use and embed in your apps than 3 Opus.

[7:20](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=440s) But 3.5 Sonnet is not all that we've released in the past week. We also released a new product feature that I think is actually more inspiring to developers in terms of thinking about and building those AI products from the ground up.

[7:32](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=452s) It's called Artifacts. Artifacts separate the content that Claude produces from the actual chat dialogue itself.

[7:40](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=460s) This allows you to work collaboratively with Claude on things from essays to SVGs to React websites.

[7:47](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=467s) Artifacts become really powerful when you combine it with 3.5 Sonnet. Those coding skills plus that reasoning ability plus that strong visual acuity enables a new product experience that's really fun to use.

[8:00](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=480s) It's also a developer's best friend in that it allows you to quickly take screenshots and Figma diagrams and quickly turn it into code and components that you can actually just go use.

[8:09](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=489s) Uh, as you can see in this, I basically cloned our entire claude.ai chat layout in React just from a single screenshot.

[8:19](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=499s) And this feature has practically been hiding in plain sight now, uh, just waiting really to be discovered for over a year and a half. Maybe this tweet is right, and we really are early on this S-curve of production- productionizing LLMs, which I think is actually pretty inspiring.

[8:37](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=517s) And Artifacts is not the only AI feature that we launched recently. On Tuesday, we released Projects.

[8:43](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=523s) Projects enables dev teams to work and collaborate much more efficiently by grounding Claude's outputs in your own knowledge, whether it's style guides or code bases or transcripts or even your past work.

[8:56](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=536s) On our Claude team plan, you can even share these projects in your chats with all your teammates.

[9:03](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=543s) At Anthropic, our engineers now upload code repos and documentation that they use, and I've started to see people actually just share the chats and the artifacts instead of Google Docs or site documentation.

[9:15](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=555s) Projects is another great example of when you think from an LLM and an AI standpoint first, you can actually start to build product experiences that complement these technologies and don't feel like a simple add-on to what you already have.

[9:29](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=569s) So now that hopefully the creative product juices are flowing in everyone's minds, I want to dive a little bit into API improvements that we've rolled out recently and things that allow you to actually build this cool stuff.

[9:41](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=581s) I also wanna give a preview of what's coming next that will enable you to build even more.

[9:47](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=587s) So a month ago, we released our new Tool Use API. Tool Use allows you to give Claude custom client-side functions that it can then intelligently leverage.

[9:57](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=597s) Tool Use also enables things like consistent structured JSON output. With 3.5 Sonnet, I've actually started to see devs give Claude hundreds of tools at a time.

[10:08](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=608s) On the developer console front, we're also continuing to iterate. We added a prompt generator that uses Claude to write prompts for you based on a task description. So you can see in this video, we put in a task description, and then out comes a optimized prompt.

[10:24](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=624s) And then once that prompt is all done, you can actually just start editing it right in the workbench itself. You can see we've also added support for variables so you can edit prompt templates as well, test things like RAG use cases.

[10:39](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=639s) And finally, we're also working on a new evaluate feature, which is currently in console right now with a beta tag, uh, and we will plan to share more on this and continue i- to iterate on it very soon.

[10:52](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=652s) So what else is next? Um, well, there's, there's two things that I can share right now. First is that you can expect more models.

[11:00](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=660s) 3.5 Haiku and 3.5 Opus are coming later this year. With each model generation, we're looking to increase the intelligence, decrease the latency, and decrease the cost. The number one thing that I tell developers is to not forget to build with that in mind.

[11:17](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=677s) Models will become smarter, cheaper, and faster in orders of months, not years. When you're planning your product roadmap, be ambitious enough to build with the belief that new models may arrive during your development period.

[11:31](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=691s) We are also working on other areas of research like interpretability.

[11:35](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=695s) In one of our latest papers called Scaling Monosemanticity, we explained how we've been able to find features within models that activate for different topics. Once you identify a feature, you're able to clamp its value and turn it up or down to actually steer the model's outputs.

[11:51](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=711s) A few weeks ago, we showed claude.ai users how this worked through Golden Gate Claude, which was a version of Claude that had the Golden Gate Bridge feature turned up significantly.

[12:03](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=723s) Whoo! [laughs] Yeah, fan favorite. We currently have a few beta testers also experimenting with a steering API. Um, this allows developers to find and clamp features for sp- specific attributes and actually turn that dial up or down, which again allows you to control Claude's outputs in a, in addition to actually just prompting it.

[12:23](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=743s) We hope to be able to roll this out to more developers in the very near future as well.

[12:28](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=748s) Now, if anything in this talk has sparked any ideas, I wanna encourage you guys to just go out there and build and make quick prototypes as fast as you can to get that validation and that feedback loop started.

[12:39](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=759s) And for even more of an incentive, uh, we actually just launched another Build with Claude contest yesterday. Whoo. It runs until July 10th. The top three projects will each receive 10K in Anthropic API credits.

[12:50](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=770s) To see more details, just visit that link below. It's just at the top of our docs page as well, so you can find it there too. I'll leave that up for a second.

[13:02](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=782s) And finally, if you have any questions or you wanna hear more about just what we're thinking about, uh, I'll be at that AWS booth down the hall for the next few hours.

[13:09](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=789s) You can also find me on X/Twitter [REDACTED:username] with two underscores. Um, I do try to read all my DMs. I spend way too much time on that site. So feel free to, uh, ask questions there as well.

[13:22](https://www.youtube.com/watch?v=EuC1GWhQdKE&t=802s) And with that, I wanna say thank you guys very much and enjoy the, uh, last day of the summit. [upbeat music]
