AI Engineer World's Fair 2024

Spreadsheets-are-all-you-need: Decoding the Decoder LLM without de code

Ishan Anand· Creator, Spreadsheets are all you need17:08

Loading slides…

Read the talk

Spreadsheets-are-all-you-need: Decoding the Decoder LLM without de code

Ishan Anand opens up a language model in Excel, follows a weekday prediction through its layers, and changes a completion by adding a vector.

From a talk by Ishan Anand

Turn sentence completion into a spreadsheet calculation

How can a computer complete Mike is quick, he moves with quickly? Ishan Anand approaches that question wearing scrubs: the audience will become AI brain surgeons, GPT-2 Small is the patient, and an Excel workbook is the operating table. His Spreadsheets are all you need project implements the model using pure Excel functions, without Python or model API calls. Anand describes more than 150 tabs and more than 124 million parameter cells. For engineers without a machine-learning degree, the demonstration offers three ways into that machinery: study its anatomy, inspect it with a virtual MRI, then intervene with a little brain surgery.

Computers already know how to complete a different kind of expression: 2 + 2 = 4. To make sentence completion computational, the model turns a word problem into a numerical one. It breaks text into tokens, maps those tokens to embeddings, and processes the resulting numbers through multi-headed attention and a multilayer perceptron, or MLP. An embedding is a vector of many numbers, not the single number used in the introductory illustration. The output also differs from elementary arithmetic: instead of one exact answer, the model produces a distribution over possible next tokens. Selecting a token and converting it back into text completes the path from language to numbers and back again.

0:240:35
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:24 · section reference included

Inspect tokens and their 768-number embeddings

The workbook makes that path navigable through its tabs. Entering the opening prompt produces quickly. For this spreadsheet's next-token calculation, Anand reports a wait of about 30 seconds. Excel is serving as an inspection tool here; he explicitly warns against using this implementation in production.

Tokenization is the first operation to inspect. The words in the initial example fit into individual tokens, but that is not a general rule. Two contrasting examples expose the difference between a tokenizer's units and a reader's intuition:

  • funology splits into fun and ology, a division that resembles meaningful word parts.
  • reinjury splits into rein and jury, rather than the intuitive prefix re followed by injury.

The algorithm selects frequent subword units. Those units can line up with morphemes, but they need not preserve a word's linguistic structure. A token is therefore neither reliably a whole word nor reliably a meaningful part of one.

Next comes the embedding lookup. Each token in GPT-2 Small has a 768-number embedding. In the displayed sheet, tokens occupy separate rows, with their embeddings beginning in column 3. The row for Mike stretches through column 770. Scrolling across that row makes the representation tangible: the model processes a long vector for each token, not a single numerical label.

3:263:38
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:26 · section reference included

Read a causal attention head and an MLP

The next tabs contain the model's layers. Attention lets a token position gather context from other positions: he can refer back to Mike, while the position containing moves can draw on the earlier quick. That context matters because quick can describe physical speed, wit, the sensitive tissue beneath a fingernail, or—in an older usage—the living. At moves, the surrounding phrase supports a continuation about movement, such as quickly or fast. The model has twelve blocks, numbered 0 through 11, each with attention followed by an MLP.

Inside one attention head, a table shows how strongly each token position attends to the others. The zeros above the diagonal encode a constraint: a position can use itself and earlier positions, but cannot look ahead. This is causal masking. Each row's attention weights sum to one, and the initial Mike position can only attend to itself. In the displayed head, the query token he assigns approximately 0.48 of its attention weight to the key token Mike. That is an interpretable relationship in this particular head; other heads are less immediately legible, and an attention weight alone does not establish a token's causal importance to the final answer.

The actual attention sheet: future positions are zero, and the “He” row assigns about 0.48 to “Mike”.Open full source frame
The actual attention sheet: future positions are zero, and the “He” row assigns about 0.48 to “Mike”.

Further down the sheet, the MLP makes another part of the computation visible. Excel's MMULT performs matrix multiplication by learned weights, and an activation function supplies the nonlinear transformation. The spreadsheet exposes these as ordinary calculations: attention gathers contextual information, and the MLP transforms the resulting representation.

5:315:42
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:31 · section reference included

Project the final state into next-token scores

The language head turns the final token position's representation into scores for possible continuations. This operation is called unembedding: a projection into the vocabulary, rather than a literal reversal of the original embedding lookup. The workbook uses greedy selection—the behavior described here as temperature zero—so its output is the highest-scoring candidate.

The displayed calculation starts with block 11's output, applies LayerNorm, then uses another MMULT with the unembedding matrix. The resulting values are logits, or raw token scores. Probabilities are a normalized interpretation of those scores; the two should not be confused. For greedy selection, the distinction does not change the winner: the largest logit also corresponds to the largest probability. The next sheet finds that maximum, obtains its token ID, and looks up the corresponding text. That finishes the forward pass for one next-token prediction.

7:498:11
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:49 · section reference included

Use the residual stream as an inspection point

Before the virtual MRI, one architectural detail changes how to picture the whole network: residual connections. Addition operations preserve an existing representation while adding contributions from attention or the MLP. Information therefore has a path around each transformation, rather than having to be replaced by that transformation's output.

Imagine the residual stream as a highway with one lane per token position. Attention moves information between lanes; the MLP transforms information within each lane. Successive layers read from and write to this shared stream. The logit lens uses that arrangement as an inspection point: apply the language head to intermediate representations and examine the candidate tokens it exposes. These projections provide a vocabulary-level view of the computation before the final layer. They are useful observations of intermediate states, rather than literal transcripts of the model's thoughts.

9:169:27
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:16 · section reference included

Follow Wednesday through the network

The MRI example begins with If today is Tuesday, tomorrow is, which the model completes with Wednesday. The talk reports correct next-day completion for all seven weekday variants of this prompt. Yet the intermediate views do not steadily announce the correct answer. At block 3, the final token position's leading candidates are not, still, and just. None is the desired weekday.

The inspection then searches for Wednesday elsewhere in the chart. At block 0, it appears among the displayed candidates at the position containing Tuesday, then disappears from the displayed lists. In later layers, time-related candidates surface: tomorrow, forever, Tuesday, and Friday. Wednesday returns in third place before rising to the leading position and remaining there. Disappearing from these short candidate lists does not mean all information about Wednesday has vanished from the residual stream.

The logit-lens sheet follows “Wednesday” through the later blocks, rising to the top of the displayed candidates.Open full source frame
The logit-lens sheet follows “Wednesday” through the later blocks, rising to the top of the displayed candidates.

The talk reports a circuit of four components sufficient for correct completion across the seven weekday variants. The components are:

  • The layer-0 MLP.
  • One attention head in layer 9.
  • The layer-9 MLP.
  • Attention in layer 10.

That component-isolation result is attributed to researchers; it is a stronger claim than simply observing the logit-lens chart. The visible trace is consistent with early processing followed by useful information becoming prominent again in later layers. Together, the components illustrate a circuit for this particular task, with different layers communicating through the residual stream.

10:2910:45
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:29 · section reference included

Find a feature that can be steered

With the anatomy and MRI complete, the demonstration moves to surgery. Its inspiration is Golden Gate Claude, the bridge-obsessed version of Claude associated with Anthropic's Mapping the mind of a large language model. The conceptual tool is a sparse autoencoder: another learned model that decomposes residual-stream activations into features whose meanings researchers investigate. Once a feature has an interpretable association, an intervention can amplify or suppress its contribution to the residual stream and change the language model's behavior.

The talk credits Joseph Bloom, Neel Nanda, and collaborators with developing sparse-autoencoder features for open models such as GPT-2 Small. The feature selected for this operation is associated with Jedi: feature 7650 in the demonstration's layer-2 autoencoder. Its index identifies a feature within that particular autoencoder, not a universal Jedi coordinate shared by language models.

12:1512:19
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:15 · section reference included

Change the completion by adding a decoder vector

The intervention begins with the Jedi feature's decoder vector. The workbook multiplies that vector by a coefficient, arranges it for injection, and adds it to the residual stream at the start of block 2. If the existing residual representation is r, the decoder direction is d_Jedi, and the coefficient is α, the operation is simply:

r′=r+αdJedir' = r + \alpha d_{\mathrm{Jedi}}

The prompt is Mike pulls out his. Without the intervention, the displayed completion is phone. Turning on the Jedi direction changes the model's internal representation while leaving those input words fixed.

Selecting Calculate now starts the spreadsheet's recalculation. While Excel works, the talk distinguishes the current operation from nearby steering methods:

  • Anthropic's feature intervention shares the idea of changing internal features, but its procedure differs from this decoder-vector addition.
  • Representation engineering, as described here, derives a steering direction using principal component analysis, or PCA.
  • Activation steering can construct a direction from contrasting activations: run the model on an input associated with what should be amplified, such as Jedi, and another associated with what should be suppressed, such as phone. Subtract the latter activations from the former and inject the resulting direction into the residual stream.

Then the calculation finishes. Against the earlier phone baseline, the completion is now lightsaber:

Fixed promptIntervention stateDisplayed completion
Mike pulls out hisJedi direction offphone
Mike pulls out hisScaled Jedi decoder vector added at block 2lightsaber

The audience cheers: the patient has become a Jedi. The successful-operation metaphor lands because the change is so concrete—the same prompt, with an added internal vector, now produces a different continuation.

The completed spreadsheet calculation: with Jedi steering turned on, “Mike pulls out his” produces “lightsaber”.Open full source frame
The completed spreadsheet calculation: with Jedi steering turned on, “Mike pulls out his” produces “lightsaber”.
13:2513:34
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:25 · section reference included

Carry architectural understanding into prompting

Opening the model helps engineers understand tool behavior and limitations, follow new research, and explain results to stakeholders without treating the system as magic. The closing example brings that architectural knowledge back to an everyday engineering task: arranging a prompt.

The talk compares a conventional transformer prompt template with one for RWKV. The RWKV template shown recommends swapping the relative order of instructions and context, because its mechanism does not access earlier information in the same way as transformer attention. This is a recommendation attached to the demonstrated template, rather than a universal rule about prompt order. It shows how a difference inside the model can affect the way an engineer supplies information at its input.

The closing handoff points to the research behind the demonstration and invites viewers to download the workbook. At the time of the recording, Anand also announces a Maven course covering the spreadsheet step by step. The workbook itself offers the immediate experiment: enter a prompt on your own machine, move through the tabs, and follow the numbers that produce its continuation.

15:2415:34
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:24 · section reference included

Resources

From the talk

  • Current tooling for sparse autoencoders; use a matching historical checkpoint before attempting to reproduce the Jedi feature.

Transcript download
Full transcript

Loading transcript…