AI Engineer World's Fair 2025

The Eyes Are The (Context) Window to The Soul: How Windsurf Gets to Know You

Sam Fertig13:37

Loading slides…

Read the talk

The Eyes Are The (Context) Window to The Soul: How Windsurf Gets to Know You

Useful coding assistance depends on more than a prompt: it combines evidence from the repository with clues about what the developer is trying to do.

From a talk by Sam Fertig

The same facts can tell different stories

What does liking Slack emojis tell you about someone? Sam Fertig begins with a handful of facts: he works at Windsurf, loves Slack emojis, sometimes wears turtlenecks, exaggerates his title, and is speaking at AI Engineer’s World’s Fair. He demonstrates the title problem immediately, correcting his introduction from principal deployed engineer to deployed engineer. Beside a neutral-face emoji, these facts offer recognition without much understanding.

Slide with Sam Fertig’s portrait and five bullets: works at Windsurf, loves Slack emojis, sometimes wears turtlenecks, exaggerates title, and speaks at AI WF. A neutral-face emoji appears above.Open full source frame
Sam Fertig’s introduction pairs five personal facts with a neutral-face emoji.

Change the neutral face to a purple devil, and the observations acquire an uncharitable interpretation. Working at Windsurf becomes an excuse to brag about Silicon Valley; enjoying Slack becomes a sign of avoiding work. Replace the devil with an angel, and the same observations support a different story.

ObservationDevil framingAngel framing
Works at WindsurfBrags about the ValleyWorks hard
Loves Slack emojisPrioritizes Slack over workValues team building
Wears turtlenecksThinks he has superior fashion senseStill has superior fashion sense
Exaggerates his titleIs power hungryIs indifferent to titles
Speaks at the conferenceLikes hearing himself talkLikes sharing with friends

The turtleneck interpretation survives both versions as the joke. Everything else illustrates how framing turns visible facts into hypotheses about behavior.

Those hypotheses still need corroboration. If trusted colleagues said Fertig spent all day on Slack, their testimony could strengthen one interpretation of his emoji habit. They do not actually give that testimony here; the example distinguishes inference from observation from supporting evidence. A coding assistant faces a similar problem when it tries to infer what a developer needs from what it can see.

1:131:28
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:13 · section reference included

Generating code is easier than making it fit

Windsurf packages its AI coding toolkit as an editor and as a plugin for multiple IDEs. Fertig presents its value proposition as making anything done in an editor faster and better. That broad promise sets up the more specific engineering problem: what information does an assistant need to make its output useful to the person sitting at the keyboard?

The opening example supplies two ways to approach that problem: make educated guesses using heuristics, and combine them with evidence. Producing code at all is already relatively accessible. Fertig estimates that someone using tools such as Windsurf or Augment could build a simple UI around a leading model, accept a prompt, and return an answer in a couple of hours or less. The harder task is generating code for a particular developer.

That code must fit an existing large codebase, follow organizational policies and standards, and respect personal coding preferences. These constraints are more specific than producing a plausible standalone answer. The slide also lists future-proofing, but Fertig explicitly drops that item rather than defining or defending it.

Slide titled “Code that...” lists fitting into an existing large codebase, adhering to organizational policies and standards, adhering to personal preferences, and being future-proof.Open full source frame
Code that fits: an existing codebase, organizational standards, personal preferences, and future-proofing.

The desired experience is an editor suggestion you accept because it matches what you intended, or a recommendation from Windsurf’s agent, Cascade, that follows your own way of thinking about the problem. Context is what connects a general ability to generate code with that particular fit.

3:574:05
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:57 · section reference included

The prompt, the codebase, and the developer

Windsurf’s context philosophy starts with two questions: what context, and how much? For the first question, Fertig returns to the distinction between heuristics and hard evidence. One bucket concerns your behavior—what you are doing and trying to accomplish. The other concerns your environment—the code and information around you.

User state includes code above and below the cursor, currently open files, files opened and then closed, and the order of those interactions. Clipboard contents and terminal activity supply additional clues. The sequence matters: the assistant is not just looking at a repository, but at how someone is moving through it. These signals help infer the task behind an explicit request.

Codebase state supplies the second bucket: repository code, documentation, rules provided by the user, memories generated by the agent, and other repository contents. Here, the environment provides material against which an inferred intention can be interpreted. A user’s explicit rule and an agent-generated memory are different sources, even though Fertig places both in this bucket.

The resulting formula is conceptual: relevant output = prompt + codebase state + user state. The prompt names the request, the codebase supplies its surroundings, and the user’s activity helps explain what they mean right now. Fertig presents this combination as part of Windsurf’s approach, leaving the proprietary details unspecified.

Slide displays “Relevant Output” above an equals sign and the expression “prompt + codebase state + user state,” with the presenter inset at lower left.Open full source frame
Relevant output = prompt + codebase state + user state.
7:017:10
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:01 · section reference included

Relevance has to arrive quickly

The second question is how much context to include. Larger context windows helped, and Fertig acknowledges that they continue to help to an extent. But capacity alone does not decide which information deserves a place in the model’s input. Fertig reports better results from optimizing the contents of the context window than from merely enlarging it. He supplies no task-specific benchmark or numerical comparison.

Selection also affects latency, which he treats as a core feature of a coding agent. Finding relevant context is difficult; finding it quickly remains difficult as the codebase grows. The practical requirement is therefore to identify useful material soon enough for the assistant to remain responsive during development.

Fertig attributes Windsurf’s strength here to its background in GPU optimization, tracing the company’s origins to ExaFunction. This is his explanation of the team’s expertise; the talk does not unpack a retrieval architecture or show how a particular GPU optimization changes retrieval time.

8:508:54
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:50 · section reference included

Ways to supply and retrieve context

Some context machinery works behind the scenes; other capabilities let users supply or direct context themselves. Fertig’s inventory spans several kinds of tools:

  • Search and structure: embedding search, plain text search, remote indexing, reranking, and AST parsing.
  • Persistent guidance and workspace context: memories, rules, and custom workspaces.
  • Additional inputs and connections: @ mentions, knowledge base, multimodal input, MCP, and Riptide.

These are named capabilities, not a demonstrated sequence of operations. Fertig explicitly describes the list as a mixture of directly relevant and tangential items, without explaining their individual implementations or what Riptide does.

10:0410:18
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:04 · section reference included

Knowing a user is a software capability

The relationship metaphor has a limit. Windsurf can use context to become more useful to you, but it remains a computer program; that does not imply a reciprocal personal relationship. Fertig shifts the invitation back to people: developers can ask the Windsurf team for clarification, and companies can discuss bringing the product into an enterprise. The talk offers those conversations rather than an enterprise deployment procedure.

11:3211:45
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:32 · section reference included

Where the context goes

An audience question about privacy brings the discussion from contextual usefulness to data handling. Fertig asks whether the concern is storage and training. The questioner’s brief clarification does not establish a more precise scope, so his answer addresses the broader issue of what is processed and where it goes.

Fertig says the processing concerns information about the user and their editor, rather than access to the wider operating machine. He also makes clear that this information leaves the editor and goes to Windsurf’s servers. Contextual assistance, in this account, includes remote processing.

He characterizes the servers as stateless and the processing as a pass-through transaction, then makes an absolute claim that Windsurf never stores or trains on user data. That statement should be read as Fertig’s account at the talk, not as a guarantee covering every product configuration. The later Windsurf Pilot Terms & Conditions, revised June 18, 2025, prohibit training on covered Customer Data but allow storage for enabled persistent cloud features, potentially abusive inputs, and profile data; third-party platforms have separate terms. Those terms postdate the conference and apply to a specific agreement. Fertig closes by directing security and legal questions to the website or the team—the place to establish which guarantees govern a particular deployment.

12:3312:34
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:33 · section reference included

Resources

Updates since the talk

Transcript download
Full transcript

Loading transcript…