AI Engineer Summit 2023
The Hidden Life of Embeddings
Loading slides…
Read the talk
The Hidden Life of Embeddings
Embedding vectors can become an interface for editing text and images: reconstruct an input, move its representation, and inspect what changes when it is decoded.
From a talk by Linus Lee
Before you start: Familiarity with text embeddings and basic vector operations is helpful; no model-training experience is required.
Getting closer to the controls
How can a reading or writing interface give people more control over a language model? Linus Lee’s experiments began with language models and traditional retrieval methods such as TF-IDF and BM25, before he joined Notion to work on AI. By the recording, Notion’s features included database AI autofill and translation, and its hiring spanned agents, tool use, evaluations, data, training, and interfaces. The product’s November 2022 announcement opened a private alpha; availability for everyone followed in February 2023.
Prompting, Lee suggests, feels like steering a car from the backseat with a pool noodle. There is a connection between your action and the vehicle’s motion, but control passes through several layers of indirection. This is especially frustrating with smaller, efficient models intended for production. Better instruction following helps, yet the interface still requires translating an intended change into words and hoping the model interprets them correctly. Looking inside the model offers a possible route to more direct control.
A text embedding might contain 1,536 or 1,024 numbers: coordinates in a high-dimensional latent space. Related spaces occur in intermediate activations, token embeddings, image models, and autoencoders. The demonstrations here focus on embedding models; extending the approach to those other representations remains a broader possibility.
Printing an embedding usually produces an unreadable wall of numbers. Yet those numbers encode features that help the model perform its task or reduce its loss. If meaningful attributes can be separated within that representation, they can become things a person observes and manipulates. A decoder supplies one way to read the representation back out; changing the vector before decoding supplies a way to intervene.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From a sentence to a vector and back
The first input describes Greg Egan’s novel Diaspora: post-human artificial intelligences, alien civilizations, and questions about reality and consciousness. Lee’s custom encoder turns that description into a 2,048-dimensional embedding. A decoder then attempts to reconstruct the input using the embedding. The resulting sentence differs from the original, but retains much of its content, including the book title. This makes the vector inspectable through the text it can produce.
Once both directions exist—text to embedding and embedding to text—the vector becomes an editable intermediate representation. Lee first increases a blur-radius control, sampling points around the original embedding and decoding them. The outputs remain related to the source but lose specific details: the correct title disappears, and the name Greg survives while referring to a different person. Nearby points produce a kind of semantic blur, rather than a dependable edit.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turning directions into editing controls
A useful edit needs a meaningful direction. Lee selects a previously computed direction associated with shorter text, moves the Diaspora embedding along it, and decodes the result. The generated descriptions become shorter while retaining roughly the same topic. Moving farther along that direction shortens them again. The control changes a continuous representation before generation, rather than issuing another verbal instruction.
That operation supports a spatial writing canvas. Lee places the description on the canvas, generates several shorter versions, clones a preferred version, and then shifts the clone toward more negative sentiment. The interface preserves alternatives as objects that can be compared and branched. Length and sentiment become navigable directions through possible versions of a passage.
The same directions can also measure differences between texts. Lee compares the objective, Wikipedia-style description with a pessimistic rewrite produced by ChatGPT. The rewrite introduces a futile search for meaning and an abyss of nihilism. Embedding both passages and projecting each vector onto the feature directions reveals where the two descriptions differ.
| Projected attribute | Pessimistic rewrite relative to the original |
|---|---|
| Negative sentiment | Farther along the negative direction |
| Artistic topic | Roughly similar |
| Length | Roughly similar |
| Vocabulary elaborateness | Appears somewhat greater |
An isolated coordinate along the negative direction does not explain much by itself. The comparison is useful because the passages share a topic while differing in tone: their relative positions expose a distinction the model represents. These projections describe the model’s features, rather than establishing an external scale of pessimism or literary quality.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Splicing two descriptions together
Next, Lee embeds two different passages: the Diaspora description and the opening of a short story about a calm, old town on the Mediterranean coast. He constructs a new vector from the first part of one embedding and the last part of the other. Describing the vector approximately as 2,000-dimensional, he takes the first thousand or so coordinates from one input and the remaining coordinates from the other.
The coordinate operation can be expressed directly in Python. Given the two encoded passages, this function preserves the first half of the Diaspora vector and replaces its second half with coordinates from the coastal-town vector:
python
import numpy as np
def splice_embeddings(
diaspora: np.ndarray,
coastal_town: np.ndarray,
) -> np.ndarray:
if diaspora.ndim != 1 or diaspora.shape != coastal_town.shape:
raise ValueError("Expected two vectors with the same shape")
midpoint = diaspora.size // 2
return np.concatenate((diaspora[:midpoint], coastal_town[midpoint:]))
The returned vector is the input to the decoder, not a combined text string. Lee calls the experiment latent-space interpolation; the specific operation demonstrated here is coordinate splicing, rather than a weighted average of complete vectors.
The decoded samples are readable mixtures rather than gibberish. They inherit aspects of topic, tone, punctuation, and structure from the inputs, including a quoted book title near the beginning. The result suggests that the representation tolerates this crude recombination well enough to produce recognizable features of both passages. It does not imply that one half of the vector exclusively stores one kind of information.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reading another model’s embeddings
A decoder trained for one embedding space need not be limited to that encoder. Lee next sends the same description through OpenAI’s ada embedding model and uses a trained linear adapter to connect its output to his decoder. The historical model name is text-embedding-ada-002, abbreviated in the talk as text-ada-002. At decoding time, the reconstruction comes from the external embedding through the adapter, without access to the source text.
The reconstruction is less faithful than the earlier example, but it retains surprising details. Diaspora remains recognizable, as does the structure of a quoted book title. The output stays broadly within the same topic, including references to rogue AI. Lee reports that rerunning the generation sometimes produces an approximately correct author name as well. Proper nouns, punctuation, structure, and topic can therefore survive in the representation well enough for this adapted decoder to recover them; the demonstration supplies examples, not a measured recovery rate.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From photograph to avatar, then from smiling to sad
The image experiments use CLIP embeddings. Lee supplies a photograph of himself and his Notion cartoon avatar as endpoints; the interface also requires a duplicate photograph for an unspecified technical reason. After the backend warms up, it generates six images bridging the photographic and cartoon representations. The sequence is imperfect but progressively more cartoon-like, with a transition Lee finds surprisingly smooth.
Because CLIP represents both text and images, text can participate in the arithmetic. Lee subtracts the embedding of a photo of a smiling man and adds the embedding of a photo of a very sad, crying man. He reduces the contribution of those text vectors based on prior experimentation, then generates another six images. This time the visible result makes him look markedly sad. The change is specified through two contrasting examples, whose vector difference supplies the edit.
The next composition is less successful. Lee adds a beach photograph and requests four outputs to save time. Generation initially appears stuck, prompting a suspicion that the interface has a bug; when he returns to the result, he acknowledges that the beach did not survive the latent-space arithmetic. Adding a vector does not guarantee that its intended visual feature will appear in the decoded image.
Across these experiments, the manipulations consist of computing example embeddings, adding or subtracting vectors, and applying normalization. Those simple operations can alter meaningful properties without a separate editing procedure for every property. Lee also shows prepared examples combining style with subject and smoothly transitioning between landscape images. The successful transformations and the missing beach belong to the same picture: these spaces offer useful structure, but the decoder’s response still has to be observed.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Making the invisible observable
These prototypes make otherwise hidden representations available to human observation. A row of floating-point values offers little intuition on its own. Reconstructing it, moving it, and comparing the results creates a way to investigate what the model represents. Direct manipulation is useful here both as an interface and as an experimental method.
Lee connects this work to the Othello world-model research and Finding Neurons in a Haystack. His own feature directions come from supplied examples and calculations involving their centroids. He contrasts that approach with then-recent work from Anthropic, Conjecture, and other labs seeking to discover feature directions without manually supplied examples. The prospect is representations that are interpretable and controllable in some useful sense—not a claim that every coordinate has an obvious human meaning.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The encoder, bottleneck, decoder, and adapter
The text system is a custom model fine-tuned from T5 as a denoising autoencoder. Its modified encoder-decoder transformer pools encoder information into a single embedding. On the decoder side, a special gated layer draws information from that embedding to generate text. The vector is therefore a bottleneck through which the information needed for reconstruction must pass. Lee leaves the finer architectural modifications to the accompanying code.
| Training path | Representation supplied to the decoder |
|---|---|
| Text reconstruction | Custom encoder → pooled embedding |
| External embedding adaptation | Other embedding model → trained linear adapter |
In the normal path, the model learns to reconstruct text from its own encoder’s embedding. In the adapted path, the linear layer maps another model’s embedding into a representation the normal decoder can use. That adapter is the bridge behind the earlier OpenAI reconstruction experiment.
Lee announces the demonstrated models on Hugging Face and points readers to a Colab notebook for interpolation and feature-interpretation experiments. The released Bottleneck T5 model family provides the access point for this work. He invites people to share interesting results and ask questions as they explore the models.
The image-generation backend is Kakao Brain’s Karlo. Lee describes it as an unCLIP diffusion model, comparable in training approach to DALL-E 2. In the demonstrated workflow, manipulated CLIP representations condition image generation, giving the image experiments a counterpart to the text decoder: a way to make the altered latent representation visible.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Representations people can work with
The design goal is to make an inscrutable representation feel small enough to fit in your hand: something you can inspect, play with, visualize, and manipulate. A shortening control, a branching canvas, or an image interpolation turns an abstract model operation into an experiment a person can follow. Those tools can deepen understanding of how models work and suggest ways to improve them.
Generative models become, in this sense, a laboratory for knowledge: a place to investigate how different modalities can be represented. Lee closes by invoking Bret Victor’s idea of a thinking medium that brings thought outside the head, into forms people can perceive and manipulate with their bodies. Building interfaces around what models actually represent could make that principle practical—giving people more humane ways to explore and work with knowledge.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Linus Lee's text autoencoder model card includes encoding and decoding examples, architecture details, checkpoint variants, and a link to interpolation experiments.
Historical specifications for text-embedding-ada-002, including its 1,536-dimensional embeddings.
Official repository for the Karlo image-generation project used in Lee's image demonstrations.
Studies how interpretable features appear in language-model activations using probes restricted to small groups of neurons.
Uses Othello move sequences to investigate learned board representations and interventions that change model predictions.
Full transcript
Loading transcript…