AI Engineer World's Fair 2026
Why Bigger Context Windows Won't Save Your Agent — Elizabeth Fuentes Leone, AWS
Loading slides…
Read the talk
Why Bigger Context Windows Won’t Save Your Agent
Elizabeth Fuentes Leone shows how to manage growing agent histories in Strands Agents: keep recent conversation, store large outputs elsewhere, share pointers between agents, and give tool work explicit limits and asynchronous handles.
From a talk by Elizabeth Fuentes Leone
At a glance
Ideas worth remembering
A larger context window does not remove the need to select information: accumulated tool output can overwhelm context and leave relevant material neglected.
Conversation managers control immediate history; long-term and graph memory let the application retrieve past information and relationships separately.
Memory pointers keep large payloads available without repeatedly placing them in context, and can be shared through invocation state across agents.
Clear tool responses guide the next decision; invocation limits bound repetition; async handles separate starting slow work from collecting its result.
Compression is useful only if the retained information still supports the task. Preserve a way to retrieve underlying evidence when a summary is insufficient.
A log-watching agent accumulates more than it needs
An agent supervises an application by retrieving its logs. Each retrieval adds another batch to the conversation history. If subsequent invocations carry that history forward, the model receives both the latest logs and the accumulated results of earlier checks. The monitoring task may stay the same while its context keeps growing. Elizabeth Fuentes Leone, an AWS developer advocate, uses this example to explain why large tool outputs can overwhelm an agent.
A larger context window gives the application more room to accumulate data, but capacity alone does not ensure that the model uses all of it well. Fuentes Leone describes an attention “U-curve”: information near the beginning and end remains easier to use, while information in the middle can be neglected. This is a qualitative failure pattern in the talk, rather than a measured threshold that applies to every model or context length. The architectural question is therefore which information should reach the model on this invocation.
Context engineering means “giving the model the information it needs when it needs it.” That includes reducing unnecessary data and considering what the data contains: tool output can bring prompt injection into the context as well as consume tokens. Curating the input serves both the answer and the amount of work required to produce it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Four ways to change what enters the context
The talk organizes context management around four complementary strategies:
- Externalize: move large data into persistent storage and retain a memory pointer that identifies it.
- Select: retrieve only the information relevant to the current task.
- Compress: summarize or compact information so it occupies fewer tokens.
- Isolate: separate context across agents, giving each agent the information its work requires.
These strategies act on different parts of the problem. Externalizing changes where data lives; selecting changes what comes back; compression changes how much detail travels; isolation changes who receives it. A log-monitoring system can combine them rather than choose a single technique for every kind of memory.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Start by managing conversation history
Strands Agents is the open-source, model-agnostic framework used for the examples. Its agent construction supplies the agentic loop behind a system prompt and a set of tools, without requiring the developer to assemble an explicit graph of nodes. The presented setup uses Amazon Bedrock; other model integrations can be supplied through model configuration.
A conversation manager is the first, relatively simple place to control growth:
- Sliding window: retain only the most recent messages. Earlier conversation falls out of the model’s immediate context.
- Summarization: condense older messages while preserving recent conversation.
- Combined management: use summaries of older material alongside retained recent messages.
The summarizing example preserves the last four messages and configures a summary ratio. The spoken explanation gives conflicting percentages—50% and then 60%—so it does not establish a consistent numeric ratio. The useful mechanism is clear: recent exchanges stay intact, while older exchanges are represented more compactly as the session continues. This gives the agent continuity without replaying every old message in full.
That convenience creates a decision about what may be lost. Keeping recent messages protects the immediate exchange, but the older summary must still carry whatever facts future answers will need. The ending returns to this limitation: a summary can be short and still omit the detail that matters.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Conversation history is only one kind of memory
Managing a session’s history does not by itself solve remembering across sessions. A new session may need something learned earlier, even though that earlier conversation is no longer present. The talk separates three forms of memory by the information they retain:
- Short-term memory: recent conversation history within the current session, managed with the conversation techniques above.
- Long-term memory: information saved for later retrieval, with a vector database offered as a storage approach. Prompts can determine which kinds of information from a conversation should be saved.
- Relationship memory: entities and their connections, represented in a graph. Fuentes Leone points to Neo4j as a place to learn more about this approach.
The three can coexist. Recent history supplies the immediate exchange; retrieval brings back relevant information from the past; graph memory retains relationships that the application wants to represent separately. Remembering something therefore need not mean carrying its entire original conversation into every future request.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Replace repeated log batches with a retrievable pointer
Return to the monitoring agent. Initially, every log retrieval enlarged the conversation. With a memory pointer, the tool retrieves the logs and stores them outside the model’s context. The agent retains an ID identifying that data. Another tool can inspect the stored logs and return a compact status, so the agent can answer a routine monitoring question without receiving the whole batch.
The observable change is in what the model receives: a pointer and a useful result replace the repeated raw log payload. If a later question requires the underlying logs, the agent passes the ID to a retrieval tool. Externalization keeps the original data available; selection determines when it returns to context. The design still depends on tools that can store, interpret and retrieve the data—the ID alone contains no diagnosis.
Where do the logs go, and what reaches the model instead? The flow below separates the large stored payload from the compact pointer and assessment. Its return path matters: the application can recover the logs when needed rather than relying entirely on an earlier summary.
In the Strands example, tools are defined with a tool decorator, and tool context exposes agent state for managing this information. The memory pointer ID remains in the context while the associated data is handled through state and storage. This makes the division between model input and tool-accessible data part of the application’s architecture.
The same division becomes useful in a multi-agent Strands swarm. Copying one agent’s full context into another gives the recipient material it may never need. Instead, the agents share a memory pointer through invocation state. They can refer to the same large dataset while keeping their individual contexts focused. Shared access to data does not require identical conversation histories.
Large logs stay in storage; context carries a pointer
Read the diagram as text
- Fetch application logs. A tool retrieves a large log batch.
- Store logs. Keep the payload outside context and identify it with a memory pointer ID.
- Inspect stored logs. A tool interprets the logs and produces a compact status.
- Agent context. Retain the pointer ID and the useful assessment.
- Retrieve when needed. Use the ID to request the underlying logs for a later question.
- Fetch application logs → Store logs: Large log payload.
- Store logs → Inspect stored logs: Logs available to the tool.
- Store logs → Agent context: Memory pointer ID.
- Inspect stored logs → Agent context: Compact status.
- Agent context → Retrieve when needed: Request logs by ID.
- Store logs → Retrieve when needed: Recover the stored logs.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give repeated tool calls an explicit stopping point
An agent can also accumulate work by invoking the same tool again and again. An unclear tool response leaves the agent without a clear basis for deciding that the work is finished. Fuentes Leone pairs two remedies: make the tool’s response clear, and impose a maximum invocation count.
The example allows each of the illustrated tools to be invoked three times. A clear response helps the model choose the next step; the count limit bounds repetition even when that choice goes wrong. The limit prevents unlimited calls to those tools, but reaching it does not establish that the task succeeded. It supplies a stopping point for execution.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Separate starting a slow job from collecting its result
Some tool problems are about elapsed time rather than context size. An MCP tool or external API may take longer than the calling application can wait. AWS API Gateway is the example of an intermediary with a maximum response time: a slow downstream operation can outlast that limit and interrupt the request.
An asynchronous handle separates starting work from retrieving its answer. One tool starts the long job; another checks its status. The application retains the handle and can collect the result on a later invocation. The presented implementation uses FastAPI. Fuentes Leone reports improved responsiveness in her tests, without specifying a measured latency reduction; whether this pattern fits depends on whether the application can return before the result is ready.
What changes in the timing of the agent’s work? The diagram shows the long operation continuing between two invocations. The first call starts the job and preserves its handle; a later status check reconnects the agent to the result. The slow work remains, but a single request no longer has to wait for its entire duration.
A handle connects two invocations to one long-running job
Read the diagram as text
- Start long job. First invocation — The first invocation initiates external work.
- Retain async handle. Between invocations — Keep a reference for a later invocation.
- External operation. Between invocations — The slow MCP or API operation produces its result.
- Check job status. Later invocation — A later invocation checks the job and retrieves its answer when available.
- Start long job → External operation: Initiate work.
- Start long job → Retain async handle: Preserve the handle.
- Retain async handle → Check job status: Resume on a later invocation.
- External operation → Check job status: Status and available result.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Preserve useful evidence without passing everything
The closing anti-patterns put a limit on enthusiasm for every technique above. Context stuffing includes everything because it is available, even when the answer needs only a small part. Indiscriminate summarization makes the opposite mistake: it reduces the history but may discard information needed to answer. Passing whole contexts between agents repeats the stuffing problem across the system.
The final cheat sheet maps concrete problems to concrete controls:
- Large log or database outputs: externalize the data, select relevant information, and keep a memory pointer.
- Several agents needing the same data: share the pointer through invocation state rather than copy the full payload into every context.
- Repeated tool calls: provide clear responses and limit invocation counts.
- Slow external work: use an asynchronous handle to retrieve the result later.
For the log-watching agent, this produces a practical division of responsibilities. Storage keeps the evidence, tools inspect and retrieve it, state carries references, and the model receives what it needs for the current answer. The important decision is what crosses into context on each invocation—and how the application recovers detail when a compact result is no longer enough.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Full transcript
Loading transcript…