Contents
  1. Representation contracts
    1. What the encoding must preserve
    2. Which object the vector represents
  2. Development
    1. Turning points in reusable representations
  3. Scores and neighborhoods
    1. Three ways to compare vectors
      1. A worked comparison
    2. What a nearest neighborhood means
  4. Learning geometry
    1. The objective chooses rewarded distinctions
    2. Pairs, negatives, and contrastive pressure
      1. From pair declarations to pressure
    3. Useful invariance without collapse
  5. Inspection and comparison
    1. What a readout can recover
    2. A task-matched comparison
  6. Larger systems
    1. Candidate retrieval, not complete search
    2. User–item compatibility is not generic similarity
    3. Alignment across modalities
  7. Failure boundaries
    1. Nearby can still be wrong
  8. Operations
    1. Version, migrate, and revalidate the space
  9. Check understanding
  10. Open questions
  11. Selected talks
  12. References
  13. Talk library
← All topics

Embeddings

An embedding turns an object—such as a token, document, image, user, or product—into a vector that another operation can compare or read. The useful question is not whether the vector captures “meaning” in general. It is whether the complete representation contract preserves the distinctions required by a particular task on the cases the system will encounter. This chapter develops that contract from vectors and objectives through evaluation, application, failure analysis, and safe migration.

Part I — Representation contracts

What the encoding must preserve

A representation is a numerical interface between an input and a later computation. An encoder produces that interface. It need not retain enough information to reconstruct the input; it needs to expose distinctions that a permitted downstream operation can use. A representation of support tickets might preserve issue type while ignoring punctuation. A representation used to reproduce exact quotations cannot safely ignore the same variation.

Preservation is therefore operational. Let an encoder map an input xx to z=f(x)z=f(x). A downstream function that turns zz into a task output is called a readout. A distinction is preserved for task TT when an appropriate readout gg can recover the required output on new cases. The claim therefore depends on the input population, the allowed readout, and the decision—not merely on whether two vectors look separated in a plot. Training receives pressure from a computable objective, which may reward only some distinctions.

Every representation is selective. An autoencoder constrained by a bottleneck or regularization must prioritize information useful for its reconstruction criterion, but low reconstruction error does not establish useful semantic neighborhoods. Conversely, a retrieval encoder can rank relevant passages without preserving exact wording. Fixed-width compression can also discard a distinction that a later, unforeseen query needs. That is why no embedding is universally semantic.

Useful invariance and harmful information loss are the same structural event judged against different tasks: changing the input causes little or no useful change in its representation. Ignoring capitalization may help topic retrieval; it may corrupt a task that must preserve legal names exactly. State the decision first, then decide which transformations should leave the representation stable.

The task decides whether invariance is useful

Example

One capitalization-insensitive representation can preserve issue type while discarding the exact-name distinction required by another readout.

“ACME refund failed”Original support-ticket text with capitalization preserved.
“Acme refund failed”A controlled variant changes capitalization but keeps the issue description.
Capitalization-insensitive encoderMaps both variants to the same task-facing representation.
Shared representationIssue information remains accessible; capitalization does not.
Issue-routing readoutUses the preserved issue distinction to select the refund queue.
Refund queueThe task decision remains stable across the controlled transformation.
Exact-name readoutAttempts to recover the original capitalization from the representation.
Exact form unavailableThe representation no longer distinguishes “ACME” from “Acme.”
Preservation is judged through a declared readout and task. Treating capitalization variants alike can be correct for issue routing while making exact-name recovery impossible.
Read the diagram as text
  • “ACME refund failed”. Original support-ticket text with capitalization preserved.
  • “Acme refund failed”. A controlled variant changes capitalization but keeps the issue description.
  • Capitalization-insensitive encoder. Maps both variants to the same task-facing representation.
  • Shared representation. Issue information remains accessible; capitalization does not.
  • Issue-routing readout. Uses the preserved issue distinction to select the refund queue.
  • Refund queue. The task decision remains stable across the controlled transformation.
  • Exact-name readout. Attempts to recover the original capitalization from the representation.
  • Exact form unavailable. The representation no longer distinguishes “ACME” from “Acme.”
  • “ACME refund failed” → Capitalization-insensitive encoder: data: original text.
  • “Acme refund failed” → Capitalization-insensitive encoder: data: capitalization variant.
  • Capitalization-insensitive encoder → Shared representation: produces the same encoding.
  • Shared representation → Issue-routing readout: readout uses issue features.
  • Issue-routing readout → Refund queue: selects queue.
  • Shared representation → Exact-name readout: readout attempts exact recovery.
  • Exact-name readout → Exact form unavailable: returns an underdetermined form.

Which object the vector represents

A vector is an ordered numerical array; its width is its dimension. Equal length makes two arrays structurally compatible, not necessarily comparable.

For text, keep several levels distinct. A token is a vocabulary unit. Its token ID is an integer name. An embedding table stores one learned row per vocabulary entry, so the ID selects a vector; the numerical size of the ID carries no semantic magnitude. A transformer then combines that initial vector with position and surrounding context, producing an occurrence-specific contextual state. The arrays and their parameters develops the shapes used in that computation.

Each row names a different represented object. Aggregation changes the unit about which later comparisons can make claims.
RepresentationRepresentsConstructed byWhat is no longer directly available
Token IDVocabulary entryTokenizer lookupMeaningful coordinate geometry
Embedding-table rowToken typeLearned lookup tableOccurrence-specific context
Contextual token stateOne token occurrenceSequence layers using surrounding inputA context-independent word vector
Pooled sequence vectorA selected span or documentMean, maximum, special-state, or learned poolingSeparate token states and some order detail
User, item, image, or region vectorThe unit selected by its encoderDomain-specific features and aggregationInput detail not retained by that contract

Even within one model family, the contract can include role prefixes, maximum input length, pooling, normalization, precision, and output truncation. E5-base-v2, for example, documents query and passage prefixes, 768-dimensional outputs, average pooling, normalization, and a 512-token limit. A vector detached from those conventions is incomplete application data.

Part II — Development

Turning points in reusable representations

Modern embeddings combine several lines of work that made different relationships computable. In 1954, Zellig Harris described words through the environments in which they occur, treating distribution as evidence about meaning rather than a complete definition of it. A 1974 vector-space retrieval report represented documents with weighted term coordinates and compared their directions. Latent semantic indexing, published in 1990 by Deerwester and colleagues, then compressed term–document associations into lower-dimensional factors, helping retrieval bridge vocabulary mismatch without assigning a human meaning to every coordinate.

Learning made the comparison rule part of fitting. The 1993 Siamese signature network trained shared encoders from labeled signature pairs. Bengio and colleagues’ 2003 neural language model learned word vectors jointly with next-word prediction, and word2vec’s 2013 continuous bag-of-words and skip-gram objectives scaled context-based learning of static word representations. Count-based and predictive methods remained related alternatives rather than successive replacements; their objective-level connection is developed later in the chapter.

The next turning points changed what an encoded object could depend on. ELMo, introduced in 2018, produced occurrence-specific word representations from surrounding text. BERT, posted in 2018 and published in 2019, used masked-token prediction to learn deeply contextual token states. In 2021, CLIP trained separate image and text encoders from paired examples, making cross-modal comparison reusable. These approaches coexist because distributional association, prediction, pair supervision, context, and cross-modal alignment preserve different relationships.

Turning points in reusable representations

  1. 1954Zellig HarrisWord environments become observable evidence about differences in meaning.Sources & context

    Contributors: Zellig Harris

    What changed: Established a distributional framing while distinguishing occurrence patterns from truth or a complete definition of meaning.

  2. 1974Vector-space retrievalWeighted term coordinates make document direction directly comparable.Sources & context

    Contributors: Gerard Salton, A. Wong, and C. S. Yang

    What changed: Turned document retrieval into geometric comparison while keeping the representation dependent on selected terms and weights.

  3. 1990Latent semantic indexingLower-dimensional factors connect terms and documents across vocabulary mismatch.Sources & context

    Contributors: Scott Deerwester and colleagues

    What changed: Compressed term–document associations so related material could compare closely without sharing every term or assigning a human meaning to each coordinate.

  4. 1993Siamese signature networkShared encoders learn compatibility from labeled signature pairs.Sources & context

    Contributors: Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Säckinger, and Roopak Shah

    What changed: Made the comparison rule trainable: genuine pairs were rewarded for small angles and forgery pairs for large angles.

  5. 2003Bengio neural language modelWord vectors are learned jointly with next-word prediction.Sources & context

    Contributors: Yoshua Bengio and colleagues

    What changed: Allowed related word representations to support generalization beyond exact sequences observed during training.

  6. 2013word2vecCBOW and skip-gram scale predictive learning of static word vectors.Sources & context

    Contributors: word2vec paper authors

    What changed: Made context-based word learning efficient through objectives that predict a word from context or context from a word.

  7. 2018ELMoA word occurrence receives a representation derived from its surrounding text.Sources & context

    Contributors: Matthew Peters and colleagues

    What changed: Changed the represented object from a context-independent word type to an occurrence-specific state assembled from a bidirectional language model.

  8. 2018 preprint; 2019 publicationBERTMasked prediction learns deeply contextual token states from both directions.Sources & context

    Contributors: Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova

    What changed: Separated input embeddings from contextual states and made one pretrained checkpoint adaptable to multiple downstream tasks.

  9. 2021CLIPPaired image and text encoders learn a reusable cross-modal score.Sources & context

    Contributors: CLIP research team

    What changed: Enabled image–text retrieval and zero-shot classification while leaving the score dependent on paired data, candidate descriptions, and the training objective.

Notice how the represented object and rewarded relationship expand—from distributional evidence and document comparison to learned pairs, prediction, contextual token states, and cross-modal alignment. These approaches coexist rather than forming a replacement ladder; spacing is not to scale.

Part III — Scores and neighborhoods

Three ways to compare vectors

A comparison rule maps two vectors to a scalar. The dot product reflects alignment and magnitude, Euclidean distance measures separation, and cosine similarity measures angular alignment for nonzero vectors.

For vectors x,y∈Rdx,y\in\mathbb{R}^d: x⋅y=∑i=1dxiyi,∥x−y∥2=∑i=1d(xi−yi)2, x\cdot y=\sum_{i=1}^{d}x_i y_i,\qquad \lVert x-y\rVert_2=\sqrt{\sum_{i=1}^{d}(x_i-y_i)^2}, cos⁡(x,y)=x⋅y∥x∥2∥y∥2. \cos(x,y)=\frac{x\cdot y}{\lVert x\rVert_2\lVert y\rVert_2}. If uu and vv are unit-normalized, then ∥u−v∥22=2−2(u⋅v)\lVert u-v\rVert_2^2=2-2(u\cdot v). Thus cosine, dot-product, and Euclidean rankings agree for unit vectors, with similarity ranked high-to-low and distance low-to-high. Cosine is undefined for a zero vector.

A worked comparison

For q=(1,0)q=(1,0), the candidates produce different rankings.
CandidateVectorDot productCosineEuclidean distance
A(2, 2)20.7072.236
B(1, 0)110
C(0, 1)001.414
RankingA, B, CB, A, CB, C, A

One query, three different rankings

Example

Dot product favors A; cosine favors B; Euclidean distance ranks C ahead of A.

Vectors from the origin

Arrows show magnitude and direction.

-0.50.2511.752.5-0.50.2511.752.5Coordinate 1 (dimensionless)Coordinate 2 (dimensionless)Query qCandidate ACandidate BCandidate Cq and BAC
  • 1. Query q
  • 2. Candidate A
  • 3. Candidate B
  • 4. Candidate C
Read coordinates and regions as data

X: -0.5–2.5 dimensionless; Y: -0.5–2.5 dimensionless, increasing up. Equal scale on both axes.

Query q (polyline)

(0, 0); (1, 0)

Candidate A (polyline)

(0, 0); (2, 2)

Candidate B (polyline)

(0, 0); (1, 0)

Candidate C (polyline)

(0, 0); (0, 1)

q and B: (1.1, 0.08)

A: (2.08, 2.08)

C: (0.08, 1.08)

Constructed vectors q=(1,0), A=(2,2), B=(1,0), and C=(0,1), shown on equal axis scales. Exact scores appear in the table.

Dot product favors A’s magnitude; cosine favors B’s matching direction; Euclidean distance favors B, then C. The task determines whether magnitude belongs in the score. Validate thresholds for the exact encoder and population because some contrastively trained spaces concentrate cosine scores in a narrow, high range.

What a nearest neighborhood means

A nearest-neighbor result is relative to a collection, representation, comparison rule, and selection policy. A kk-nearest query returns a fixed count even when every candidate is poor; a radius query can return none or many. Ties at a boundary can make results depend on ordering. “Nearest” therefore means best among the supplied candidates under a declared rule, not relevant in an application sense.

High-dimensional geometry needs diagnostics rather than folklore. Anisotropy is directional concentration: random items can share a large background cosine because much of the space points along common directions. Hubness is different: a hub appears unusually often in other points’ neighbor lists. Distance concentration concerns relative distance spread. These effects depend on the learned distribution and metric; high dimension alone does not prove any of them.

An original-space neighborhood audit should make the contract and boundary cases inspectable.
FieldQuestion answered
Candidate ID and sourceWhat object was compared?
Raw score and rankWhat did the named metric return?
Tie and boundary statusCould ordering change at the cutoff?
Neighbor occurrence countDoes this item act as a hub?
Human or task relevanceWas geometric proximity useful?

Two-dimensional projections are inspection aids, not evidence about the original space. t-SNE layouts can change with perplexity, optimization, and random runs; displayed cluster distances and apparent density can be misleading. UMAP can introduce apparent tears and responds to neighborhood and packing parameters. Always compute authoritative neighbors in the original representation, then label any projection as a separate derived representation.

Axes are fragile interpretations: rotations can change coordinates while preserving comparisons. Some objectives also permit prediction-preserving reparameterizations that alter cosine geometry. Naming a coordinate requires separate evidence.

Part IV — Learning geometry

The objective chooses rewarded distinctions

A representation’s geometry emerges from what fitting rewards. A predictive objective preserves features useful for predicting context or successors. A reconstruction objective preserves detail that reduces reconstruction loss. Classification rewards boundaries useful for supplied labels. Metric and contrastive objectives directly compare selected examples. These labels describe pressure, not a complete geometry: architecture, data, capacity, augmentation, regularization, and optimization also matter.

Ask what must be recoverable to reduce each objective, rather than assuming every objective learns the same notion of meaning.
Objective familyTraining relationshipWhat it directly rewards
Context predictionWord and surrounding wordsFeatures useful for predicting observed contexts
Masked predictionVisible input and hidden targetFeatures useful for recovering selected missing content
ReconstructionInput and reconstructionInformation favored by the loss and bottleneck
ClassificationInput and supplied labelFeatures supporting the trained decision boundary
Metric or contrastiveAnchor, related items, alternativesRelative compatibility under declared pairs

The historical divide between count and prediction can be overstated. Skip-gram with negative sampling admits a matrix-factorization interpretation under stated assumptions, and controlled work transferring smoothing and context choices between count-based and predictive representations found no consistently superior family across its tested tasks. Preprocessing and evaluation choices can explain apparent family-level advantages.

Pairs, negatives, and contrastive pressure

In contrastive learning, the training procedure declares a positive pair related for the task and supplies competing alternatives, often called negatives. The names describe training roles, not objective truth. Two augmented views of one image may be positive for object recognition, while two camera views must remain distinguishable for a viewpoint-estimation task.

For anchor ii, positive jj, candidate set A(i)A(i), similarity ss, and temperature τ>0\tau>0, a representative contrastive loss is ℓi,j=−log⁡exp⁡(s(zi,zj)/τ)∑k∈A(i)exp⁡(s(zi,zk)/τ). \ell_{i,j}=-\log\frac{\exp(s(z_i,z_j)/\tau)}{\sum_{k\in A(i)}\exp(s(z_i,z_k)/\tau)}. The numerator rewards the positive’s score relative to every candidate in the denominator. Lower τ\tau sharpens score differences; it does not turn the result into a calibrated probability of real-world relevance.

From pair declarations to pressure

Illustrative pseudocode

Python-like pseudocode
for anchor, positive in batch:
    candidates = other_views_in_batch
    scores = [cosine(encode(anchor), encode(x)) / temperature
              for x in candidates]
    positive_index = index_of(positive, candidates)
    loss += cross_entropy(scores, positive_index)
# Correct only if every competing candidate is truly negative
# for the relationship this task intends to learn.

A margin or triplet loss asks an anchor–positive distance to beat an anchor–negative distance by a margin. InfoNCE-style losses compare a positive with many sampled alternatives. In-batch negatives are efficient because another pair’s positive can serve as a competitor, but the shortcut is only valid when those items are genuinely negative for the intended relation. Same-class or otherwise compatible examples create false negatives, teaching the model to separate things the application wants together.

A false negative turns efficient batching into harmful pressure

Example

An in-batch item can be relevant under the application relation yet still be pushed away because the loss assigned it the role of competitor.

1 / 2 · Declare batch roles

The anchor has one declared positive. Every other candidate is treated as a competitor, including a second document that is relevant under the application relation.

New this stepAnchor: password resetA query seeking instructions for resetting an account password.
New this stepDeclared positiveThe labeled password-reset guide paired with the anchor.
New this stepValid competitorAn unrelated billing-address article from another pair.
New this stepFalse negativeA relevant account-recovery guide that appears as another pair’s positive.
New this stepIntended task relationBoth recovery documents are useful answers for the anchor.
“Positive” and “negative” are training roles. The false negative belongs with the anchor under the intended task, but its position in another batch pair makes the loss lower its relative score.
Read the diagram as text
  • Anchor: password reset. A query seeking instructions for resetting an account password.
  • Declared positive. The labeled password-reset guide paired with the anchor.
  • Valid competitor. An unrelated billing-address article from another pair.
  • False negative. A relevant account-recovery guide that appears as another pair’s positive.
  • Intended task relation. Both recovery documents are useful answers for the anchor.
  • In-batch contrastive loss. Rewards the declared positive relative to every other candidate in the batch.
  • Pull declared pair together. Training raises the declared positive’s score relative to competitors.
  • Push valid competitor away. Lower compatibility agrees with the intended retrieval relation.
  • Push relevant item away. Lower compatibility conflicts with the intended retrieval relation.
  • Anchor: password reset → Declared positive: batch declares positive.
  • Anchor: password reset → Valid competitor: batch supplies competitor.
  • Anchor: password reset → False negative: batch supplies competitor.
  • Anchor: password reset → Intended task relation: task asks for recovery help.
  • Declared positive → Intended task relation: satisfies intended relation.
  • False negative → Intended task relation: also satisfies intended relation.
  • Declared positive → In-batch contrastive loss: occupies positive slot.
  • Valid competitor → In-batch contrastive loss: enters denominator.
  • False negative → In-batch contrastive loss: enters denominator.
  • In-batch contrastive loss → Pull declared pair together: raises relative score.
  • In-batch contrastive loss → Push valid competitor away: lowers relative score.
  • In-batch contrastive loss → Push relevant item away: lowers relative score.
  1. Declare batch roles. The anchor has one declared positive. Every other candidate is treated as a competitor, including a second document that is relevant under the application relation. Active: Anchor: password reset, Declared positive, Valid competitor, False negative, Intended task relation. New: Anchor: password reset, Declared positive, Valid competitor, False negative, Intended task relation.
  2. Apply relative-score pressure. The loss pulls the declared pair together and pushes both competitors down. That pressure is appropriate for the billing article but conflicts with the task relation for the account-recovery guide. Active: Anchor: password reset, Declared positive, Valid competitor, False negative, Intended task relation, In-batch contrastive loss, Pull declared pair together, Push valid competitor away, Push relevant item away. New: In-batch contrastive loss, Pull declared pair together, Push valid competitor away, Push relevant item away.

Hard-negative mining concentrates training on high-scoring confusions, but “hard” should mean confusable and still incorrect. An unlabeled relevant item is not a useful negative merely because the current model scores it highly. Pair construction, sampling, and augmentation jointly define the relation the space is trained to expose.

Useful invariance without collapse

Invariance means a task-approved transformation leaves the relevant representation or decision effectively stable. Equivariance means the representation changes in a predictable corresponding way. Translating an image might preserve its category while shifting a spatial feature map. Cropping an image and segmentation target at the same coordinates preserves their relationship; cropping only the image corrupts it.

The required behavior depends on the task. A local descriptor used for correspondence must retain enough spatial variation to distinguish locations, while a category representation may benefit from ignoring small position changes. Local features and correspondence gives the concrete visual case. Augmentation is thus a proposed task contract, not a universally harmless way to create more data.

A contrastive agreement objective has a trivial solution if every input maps to the same vector. This is representation collapse: intended positives agree, but no inputs remain distinguishable. VICReg separates three pressures: match paired views, maintain variation across examples in each coordinate, and reduce redundant covariance across coordinates. The construction distinguishes constant-output collapse from repeated information across dimensions.

Avoiding collapse is necessary but insufficient. Nonzero variance does not show that the space preserves identity, negation, relevance, or any other required relation. Test the desired invariances and separations directly on held-out transformations, including changes that should remain stable and nearby changes that must alter the decision.

Part V — Inspection and comparison

What a readout can recover

A readout maps a representation to a task output. Neighbor inspection asks whether nearby examples look useful. A linear probe fits a weighted-sum classifier while keeping the encoder frozen. A nonlinear probe permits a richer decision boundary. Fine-tuning changes the representation itself and therefore answers a different question; adapting reusable features develops those choices.

Held-out linear-probe success supports a bounded claim: the tested distinction is accessible to that linear predictor on that population under that training procedure. It does not prove that each coordinate has a human meaning, that the original model uses the distinction, or that the same boundary survives distribution shift. Successive network layers can improve linear separability without adding information about the original input.

The readout changes what is accessible

Example

A straight boundary solves one target on fixed points, while an XOR-like target needs a richer readout; neither layout alone proves causal use by the encoder.

Target A: linearly accessible

Positive examples lie to the right of the dashed linear boundary.

-2-1012-2-1012Representation coordinate 1 (dimensionless)Representation coordinate 2 (dimensionless)Target A positiveTarget A negativeLinear boundary
  • 1. Target A positive
  • 2. Target A negative
  • 3. Linear boundary
Read coordinates and regions as data

X: -2–2 dimensionless; Y: -2–2 dimensionless, increasing up. Equal scale on both axes.

Target A positive (points)

(1, 1); (1, -1)

Target A negative (points)

(-1, 1); (-1, -1)

Linear boundary (polyline)

(0, -1.8); (0, 1.8)

Target B: richer readout needed

Diagonal positives form an XOR-like target that one line cannot separate.

-2-1012-2-1012Representation coordinate 1 (dimensionless)Representation coordinate 2 (dimensionless)Target B positiveTarget B negative
  • 1. Target B positive
  • 2. Target B negative
Read coordinates and regions as data

X: -2–2 dimensionless; Y: -2–2 dimensionless, increasing up. Equal scale on both axes.

Target B positive (points)

(1, 1); (-1, -1)

Target B negative (points)

(-1, 1); (1, -1)

Constructed points shown on shared equal scales. The encoder output is treated as fixed. In the first panel, target labels separate by the sign of x. In the second, the same four locations receive an XOR-like labeling that no single line separates.

A powerful probe complicates interpretation because it can learn substantial task logic itself. Conversely, probe failure does not prove absence; another readout may recover the property. Attribute-recovery studies also show why testing one weak attacker is inadequate: demographic information remained recoverable from some text representations even after the training adversary approached chance. Probe capacity, train/test separation, baselines, and overfitting checks belong in the claim.

A decoder is another readout. Linus Lee demonstrated a custom text autoencoder whose decoder approximately reconstructed text from a pooled representation, and a learned adapter enabled approximate recovery from another model’s embeddings. That does not make arbitrary embedding APIs invertible, but it illustrates that vectors can retain details beyond the application’s intended use. Stored embeddings should therefore be included in sensitive-data inventories.

A task-matched comparison

Choosing a representation is an evaluation problem. Begin by specifying the represented unit, intended population, query or readout operation, frozen or adapted parameters, preprocessing, pooling, normalization, metric, baselines, held-out split, resource limits, and consequential slices. Generalization requires new cases from the intended setting; matched comparisons keep cases aligned across candidates.

Keep unlike claims and outcomes separate rather than collapsing them into one representation-quality score.
EvidenceWhat it supportsWhat it does not establish
Annotated pair similarityFit to the benchmark’s relationRetrieval, classification, or product value
Frozen linear probeLinear accessibility on held-out casesCausal use or nonlinear absence
Recall at kCoverage of labeled relevant itemsPrecision, answer quality, or index latency
End-to-end task resultBehavior of the complete tested systemAttribution to one changed component
Operational measurementsCost, latency, and reliability under a workloadSemantic fitness outside that workload

MTEB deliberately evaluates several readouts, including classification on frozen vectors, clustering, pair scoring, and retrieval. Its original results were strongly task-dependent: strength on semantic textual similarity did not guarantee strength on retrieval. BEIR likewise found dense retrievers strong on some domains and below BM25 on others. A public aggregate is useful evidence about its stated tasks, not a portable ordering for every application.

A 2025 Weights & Biases chatbot evaluation illustrates the decision rule. Four embedding models were compared with recall at ten on logged queries and generated queries. The desired validation was not only similar scores but preservation of model ordering. In that application, the reported winner differed from what public English MTEB standing or the incumbent model might have suggested. Exact scores, sample size, and uncertainty were not supplied, so the lesson is the comparison design rather than a current model ranking.

Retrieval metrics can also trade off. In a fifty-task code experiment, windowed reads plus semantic search reportedly improved precision, while broader default exploration retained better file recall and the two windowed conditions had similar recall. Because windowing itself changed behavior, only the matched windowed conditions isolate the incremental semantic-search question. Neither precision nor recall alone established downstream solution quality.

Part VI — Larger systems

Candidate retrieval, not complete search

A dense retriever factorizes candidate comparison. Corpus units are encoded and indexed before requests. A new query is encoded at request time, then its vector is compared with compatible corpus vectors. Dense Passage Retrieval used separate question and passage encoders with dot-product scoring; training contrasted relevant passages with negatives. This makes broad candidate generation efficient because passage computation is reused.

Separate encoders can learn asymmetric roles: a query asks for information while a document supplies it. Some libraries therefore distinguish query and document encoding methods or prefixes. Shared vector width and callable APIs do not prove role compatibility; preprocessing and training conventions must match the stored corpus.

The embedding component stops at candidates. Similarity is not a probability that a passage proves an answer, and fixed top-k retrieval has no built-in rejection criterion. Lexical matching can outperform dense retrieval under some shifts, while metadata constraints, permissions, freshness, hybrid retrieval, reranking, and answer construction remain separate responsibilities developed in Search and Retrieval.

A bi-encoder also moves repeated work outside the request path. Pinterest described computing pin embeddings offline and query embeddings online, refreshing a pin only when its inputs meaningfully changed. This scales differently from a cross-encoder that jointly processes each query–document pair. A common pattern is broad bi-encoder retrieval followed by a cross-encoder reranker over a much smaller set.

Dense retrieval ends at candidates

Corpus encoding is reusable offline work; query encoding and comparison happen online, while relevance, permissions, and answer decisions remain downstream.

Corpus unitsDocuments, chunks, images, or other explicitly selected candidate units.
Document encoderApplies the versioned candidate-side representation contract.
Vector indexStores candidate vectors with identifiers and source metadata.
Incoming queryThe current information request.
Query encoderApplies the compatible query-side role and preprocessing.
Scored candidatesNearest or highest-scoring items under the configured metric.
Search decision stagesLexical signals, metadata, permissions, reranking, rejection, and response logic.
Documents are encoded and indexed before requests. At request time, a compatible query encoder produces a vector, the index returns scored candidates, and later search components decide whether those candidates are relevant and permitted. Data and control edges are labeled separately.
Read the diagram as text
  • Corpus units. Documents, chunks, images, or other explicitly selected candidate units.
  • Document encoder. Applies the versioned candidate-side representation contract.
  • Vector index. Stores candidate vectors with identifiers and source metadata.
  • Incoming query. The current information request.
  • Query encoder. Applies the compatible query-side role and preprocessing.
  • Scored candidates. Nearest or highest-scoring items under the configured metric.
  • Search decision stages. Lexical signals, metadata, permissions, reranking, rejection, and response logic.
  • Corpus units → Document encoder: data: candidate content.
  • Document encoder → Vector index: data: vectors and IDs.
  • Incoming query → Query encoder: data: request text.
  • Query encoder → Vector index: control: search with query vector.
  • Vector index → Scored candidates: data: scores and IDs.
  • Scored candidates → Search decision stages: data: candidate set.

User–item compatibility is not generic similarity

Recommendation systems can learn user and item embeddings from interaction data. In matrix factorization, their dot product approximates observed feedback. Missing entries in the interaction matrix are not automatically dislikes: fitting only positives admits trivial behavior, while treating every missing entry equally as negative can overwhelm the observed signal. The objective determines what compatibility means.

Content-based recommendation answers a different question. It places item attributes and user preferences in a shared feature space, allowing a new item to be scored through its properties before it has extensive interaction history. Collaborative representations instead learn from behavioral structure. Systems often combine them because item-ID embeddings alone struggle with cold-start items.

The same mathematical operation can serve distinct relations.
ComparisonQuestionTypical evidence
Item ↔ itemWhich items resemble or co-occur with this item?Content features, graph structure, co-purchases
User ↔ itemWhich items are compatible with this user state?Interactions, preferences, context
Query ↔ itemWhich items match this request?Query–item logs, content, request context

Magnitude can carry an objective-dependent signal; cosine normalization removes it. A high user–item score does not mean an item resembles a person. Exposure, exploration, long-term satisfaction, and product value belong to Recommendation Systems.

Alignment across modalities

Cross-modal embeddings make different input types comparable by learning the relationship, not by declaring their vectors the same width. CLIP uses separate image and text encoders and trains matched image–caption pairs to score above mismatches. At inference, text descriptions can be compared with an image for retrieval or zero-shot classification. The score expresses learned correspondence under the paired data and objective, not a symbolic inventory of visual facts.

This distinction matters operationally. A drawing game used CLIP cosine similarity to rank images against arbitrary prompts, enabling open-set comparisons beyond a fixed class list. Developers also reported that players could score well by writing prompt words in the image, prompting a handwriting-related penalty. The representation had learned a correspondence useful for the score but different from the application’s intended notion of a good drawing.

Cross-modal retrieval can preserve coarse subject alignment while missing order, attribute binding, and relations. ARO evaluates captions that retain constituent words but reverse relations, reassign attributes, or reorder content; tested image–text models showed weaknesses even when ordinary retrieval remained strong. Success at finding a broadly related image does not establish who did what to whom.

Some systems encode screenshots containing text, images, and layout, but the retrieved vector is still only an index key. The application must resolve the stored image and pass the actual visual content to a multimodal model for interpretation. Detailed modality fusion belongs to Multimodal Models and Applications, while visual representation mechanisms belong to Computer Vision.

Part VII — Failure boundaries

Nearby can still be wrong

Semantic similarity fails when the representation exposes a relation different from the decision the application needs. Common mechanisms include objective mismatch, missing context, wrong represented unit, lossy pooling, false negatives, domain or language shift, role-format mismatch, hubs, and unstable cutoff neighbors. Diagnose the mechanism instead of treating every bad result as a vague embedding-quality problem.

Each row changes a consequential relation while preserving much of the surface content. Scores and failure rates depend on the tested model; the table states the supported failure pattern, not a universal threshold.
ChangeWhy vectors may remain closeWhat the application can get wrong
Synonym ↔ antonymOpposites often occur in similar contextsApproved versus denied, strong versus weak
Affirmation ↔ negationMost tokens and topic remain sharedSupported versus rejected claim
Number replacementSentence structure and vocabulary remain similarQuantity, date, or dosage
Relation or attribute reassignmentObjects and attributes remain presentWho owns, wears, or acts on what
Associated ↔ genuinely similarRelated words co-occur without being substitutableCategory or equivalence decision
Same topic ↔ entailmentA hypothesis can mention the same entities without following from the textWhether evidence supports a conclusion

Many application relations are directional or structured. “Answers this question,” “newer than,” “caused by,” and “permitted for” cannot be established by an undirected proximity score alone. Graph traversal can start from a semantic match and then follow stored ownership or other edges, but the relationship must exist and be queried explicitly. Current permission requires an authority check, as explained in Enforce current authority.

Similarity is also not identity. Near-duplicate candidates can be proposed geometrically, but merging records requires an identity rule and provenance. Duplicates depend on identity develops that decision. Likewise, a memory fact can be topically close yet operationally irrelevant: a dog named Melody may match a request about favorite tunes without expressing any listening preference.

Test these failures with minimal pairs and slices. Change one entity, number, negation, relation, language, domain, or input convention while holding other content fixed. Measure both aggregate task performance and the decision-sensitive slice. A model improved for negation can still become worse on another category, so one repaired distinction does not certify general semantic fidelity.

Part VIII — Operations

Version, migrate, and revalidate the space

Once vectors are stored, an embedding becomes a versioned data dependency. The contract includes encoder checkpoint, tokenizer and preprocessing, query or document role instructions, represented unit, truncation, pooling, output layer, dimension, precision, normalization, similarity metric, source revision, and index schema. Pinning only the model name or checking array shape leaves important behavior unspecified.

Equal dimensions do not align independently trained spaces. Google explicitly documents that vectors from gemini-embedding-001 and gemini-embedding-2 cannot be compared directly and that migration requires re-embedding stored data. Switching only the online query encoder can therefore silently compare a new query space with old candidate vectors, producing structurally valid but meaningless rankings.

A safe migration coordinates the whole serving bundle.

  • Preserve sources — Keep source records and lineage needed to generate the new vectors; old vectors alone cannot be re-encoded.
  • Build separately — Create a new collection or named-vector generation with its own dimension, metric, identifiers, and metadata.
  • Catch up changes — Coordinate concurrent upserts, partial updates, and deletions so validation does not compare a stale generation.
  • Evaluate matched work — Run old and new bundles on the same held-out queries, failure slices, and operational workload; choose workload-specific acceptance criteria.
  • Cut over together — Switch the query encoder, preprocessing, scoring rule, and candidate index as one compatible release. If an alias performs the index switch, remove the old target and add the new target in one aliases request, inspect every action result, and use must_exist where appropriate so a failed removal cannot leave an unintended target.
  • Retain rollback — Keep the prior compatible bundle until rollback safety, writes, deletions, and permission changes have been reconciled.

A migration keeps two spaces separate until cutover

Source records produce versioned candidate spaces; the query encoder and serving target switch to the compatible generation only after catch-up and matched validation.

1 / 4 · Serve the old bundle

The stable alias has one intended target, the v1 index, and online queries use encoder v1.

New this stepPreserved source recordsVersioned content and lineage from which vectors can be regenerated.
New this stepOld vector generationCandidate vectors produced by encoder contract v1.
New this stepQuery encoder v1Online query path compatible with the old generation.
New this stepStable serving aliasApplication-facing name with exactly one intended active generation.
New this stepAlias → v1 servingThe alias and query encoder v1 form the active compatible bundle.
A four-step blue–green migration. Stable entity IDs persist across steps. The status nodes show the single serving generation in each state; no edge implies that the alias targets both indexes. At cutover, one aliases request removes the old target and adds the new target, action failures are inspected, and `must_exist` is used where appropriate. The old compatible bundle remains available for rollback.
Read the diagram as text
  • Preserved source records. Versioned content and lineage from which vectors can be regenerated.
  • Old vector generation. Candidate vectors produced by encoder contract v1.
  • New vector generation. Candidate vectors produced separately by encoder contract v2.
  • Query encoder v1. Online query path compatible with the old generation.
  • Query encoder v2. Online query path compatible with the new generation.
  • Stable serving alias. Application-facing name with exactly one intended active generation.
  • Alias → v1 serving. The alias and query encoder v1 form the active compatible bundle.
  • v2 building, not serving. Backfill and concurrent updates are still being applied.
  • v2 validated, not serving. Matched task, slice, operational, and update-completeness checks passed.
  • Alias → v2 serving. The alias and query encoder v2 switched as one compatible release.
  • v1 bundle retained. The prior compatible bundle remains available while rollback is safe.
  • Preserved source records → Old vector generation: data: encode with v1.
  • Preserved source records → New vector generation: data: encode with v2.
  • Query encoder v1 → Old vector generation: compatible comparisons.
  • Query encoder v2 → New vector generation: compatible comparisons.
  1. Serve the old bundle. The stable alias has one intended target, the v1 index, and online queries use encoder v1. Active: Preserved source records, Old vector generation, Query encoder v1, Stable serving alias, Alias → v1 serving. New: Preserved source records, Old vector generation, Query encoder v1, Stable serving alias, Alias → v1 serving.
  2. Build and catch up v2. Preserved sources and concurrent changes populate a separate v2 generation without cross-version scoring or serving it through the alias. Active: Preserved source records, Old vector generation, New vector generation, Query encoder v1, Query encoder v2, Stable serving alias, Alias → v1 serving, v2 building, not serving. New: New vector generation, Query encoder v2, v2 building, not serving.
  3. Validate matched work. The v2 query encoder and v2 index are evaluated together on held-out tasks, failure slices, latency, and update completeness while v1 remains the sole serving target. Active: Preserved source records, Old vector generation, New vector generation, Query encoder v1, Query encoder v2, Stable serving alias, Alias → v1 serving, v2 validated, not serving. New: v2 validated, not serving.
  4. Cut over together. One aliases request removes v1 and adds v2; action failures are inspected and `must_exist` is used where appropriate. Online preprocessing and the query encoder switch to v2 in the coordinated release, while the complete v1 bundle remains available for rollback. Active: Preserved source records, Old vector generation, New vector generation, Query encoder v1, Query encoder v2, Stable serving alias, Alias → v2 serving, v1 bundle retained. New: Alias → v2 serving, v1 bundle retained.

An alias can give the application a stable index name. Removing the old target and adding the rebuilt target in one aliases request can switch that name atomically, but the caller must inspect action failures and use must_exist where appropriate to prevent an unintended partial action list. Alias atomicity still does not transact an external query encoder, caches, or configuration, so release orchestration must coordinate those artifacts. After cutover, monitor task metrics and important slices rather than vector statistics alone; distribution and traffic changes can invalidate an earlier acceptance result.

The governing rule is simple: an embedding is not merely an array. It is a task-shaped measurement produced by a versioned procedure. Its geometry becomes useful only when the represented object, training relationship, comparison rule, and downstream evidence remain compatible.

Open questions

  1. Can representation objectives be compared under genuinely controlled conditions—holding architecture, data, augmentations, capacity, and optimization fixed—so engineers can isolate which objective creates a useful distinction rather than attributing a whole-system difference to the loss name?

  2. How can a system detect that a deployed embedding space has lost a consequential relation before aggregate retrieval metrics move? Progress would combine maintained minimal-pair slices, traffic coverage, neighbor diagnostics, and downstream decision outcomes without treating vector drift alone as failure.

  3. How should contrastive systems discover false negatives at production scale when relevance is incomplete or changes by user and task? Useful progress would identify mislabeled competitors without collapsing hard-negative training into expensive universal pairwise judgment.

  4. How can representation migrations preserve continuous updates, deletions, permissions, and rollback across indexes, encoders, caches, and external configuration? Existing blue–green and alias mechanisms cover important pieces, but acceptance thresholds and cross-system transactions remain workload-specific.

  5. What privacy controls are appropriate for embeddings whose source text may be partly reconstructable? Progress would measure recovery risk for the exact encoder and access model, then connect retention, access, deletion, and incident response to both vectors and their source lineage.

Follow the curated reading path through the speakers and demonstrations behind this entry.

18 min

AI Engineer Summit 2023 · 2023

The Hidden Life of Embeddings

Linus Lee

Cited in this entry

Explores reconstruction, feature directions, and cross-model decoding, making both the information retained by embeddings and the limits of interpreting coordinates concrete.

Watch talk

Explore more talks

The rest of the library, beyond the curated path. Cited talks support this entry; reviewed transcripts were processed in full. Metadata candidates have not been reviewed as sources or verified as topic members.

25 matching talks

TalkSpeakerEventYear
Ishan AnandAI Engineer World's Fair 20252025
Jack MorrisAI Engineer Code 20252025
Daniel HanAI Engineer World's Fair 20242024
Sander DielemanAI Engineer Europe 20262026
Isaac RobinsonAI Engineer Europe 20262026
Eugene YanAI Engineer World's Fair 20252025
Anton TroynikovAI Engineer Summit 20232023
Kevin HouAI Engineer World's Fair 20242024
Raia HadsellAI Engineer Europe 20262026
Yesu FengAI Engineer World's Fair 20252025
Ishan AnandAI Engineer World's Fair 20242024
Angelos PerivolaropoulosAI Engineer Europe 20262026
Jonathan FernandesAI Engineer World's Fair 20252025
Andreas Kolleger, Zach Blumenthal, Michael Hunger, TomaszAI Engineer World's Fair 20242024
Peter RobicheauxAI Engineer World's Fair 20252025
Shafik QuoraisheeAI Engineer World's Fair 20252025
Stop Using RAG as Memory

Cited in this entry

Daniel ChalefAI Engineer World's Fair 20252025
Devansh TandonAI Engineer World's Fair 20252025
Andreas KolleggerAI Engineer World's Fair 20252025
Git push, get an AI API.

Transcript reviewed

Ryan Fox-TylerAI Engineer World's Fair 20242024
Frank LiuAI Engineer World's Fair 20252025
Shivam VermaAI Engineer World's Fair 20252025
Apoorva JoshiAI Engineer World's Fair 20252025
Freddy LecueEvent unknownUnknown
Vinoo GaneshEvent unknownUnknown

References

Coverage and source review
Processed transcripts
28 processed in full · 5 in the curated path
Automated source review
Passed
Metadata candidates
2 unreviewed; not verified topic membership
Corpus version
1bd8e407b26a07b33815594e1b2db5f41827119a2b3cb6fbf240f9fc571fc767

Automated review checks source support; it is not publication approval.

A synthesis of selected conference talks and technical references. Citations link to the source material; they do not imply that every talk on this subject is included.

  1. Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares

    https://web.stanford.edu/~boyd/vmls/vmls.pdf

  2. Understanding Intermediate Layers Using Linear Classifier Probes

    https://arxiv.org/html/1610.01644

  3. Deep Learning, Chapter 15: Representation Learning

    https://www.deeplearningbook.org/contents/representation.html

  4. Deep Learning, Chapter 14: Autoencoders

    https://www.deeplearningbook.org/contents/autoencoders.html

  5. Embeddings are Stunting Agents: How Codeium Breaks Through the Ceiling for Retrieval

    https://www.youtube.com/watch?v=DuZXbinJ4Uc&t=271

  6. Dimensionality Reduction by Learning an Invariant Mapping

    https://cs.nyu.edu/~sumit/publications/assets/cvpr06.pdf

  7. This Is Not Correct! Negation-aware Evaluation of Language Generation Systems

    https://aclanthology.org/2023.inlg-main.12.pdf

  8. Gemini API Embeddings documentation

    https://ai.google.dev/gemini-api/docs/embeddings?authuser=2&hl=en

  9. PyTorch Embedding

    https://docs.pytorch.org/docs/2.13/generated/torch.nn.modules.sparse.Embedding.html

  10. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

    https://aclanthology.org/N19-1423.pdf

  11. How LLMs work for Web Devs: GPT in 600 lines of Vanilla JS

    https://www.youtube.com/watch?v=ZuiJjkbX0Og&t=2016

  12. Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks

    https://aclanthology.org/D19-1410.pdf

  13. Sentence Transformers: SentenceTransformer API

    https://www.sbert.net/docs/package_reference/sentence_transformer/model.html

  14. How Contextual Are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings

    https://aclanthology.org/D19-1006.pdf

  15. Google Recommendation Systems: Matrix factorization

    https://developers.google.com/machine-learning/recommendation/collaborative/matrix

  16. Learning Transferable Visual Models From Natural Language Supervision

    https://arxiv.org/html/2103.00020

  17. E5-base-v2 Model Card

    https://huggingface.co/intfloat/e5-base-v2/blob/main/README.md

  18. Distributional Structure

    https://corpus.bfsu.edu.cn/Zellig_Harris_Distributional_Structure_Word_1954.pdf

  19. A Vector Space Model for Automatic Indexing

    https://files.eric.ed.gov/fulltext/ED096986.pdf

  20. Indexing by Latent Semantic Analysis

    https://www.cs.csustan.edu/~mmartin/LDS/Deerwester-et-al.pdf

  21. Signature Verification using a "Siamese" Time Delay Neural Network

    https://papers.nips.cc/paper/1993/file/288cc0ff022877bd3df94bc9360b9c5d-Paper.pdf

  22. A Neural Probabilistic Language Model

    https://jmlr.csail.mit.edu/papers/volume3/bengio03a/bengio03a.pdf

  23. Efficient Estimation of Word Representations in Vector Space

    https://arxiv.org/pdf/1301.3781

  24. Neural Word Embedding as Implicit Matrix Factorization

    https://proceedings.neurips.cc/paper_files/paper/2014/file/b78666971ceae55a8e87efb7cbfd9ad4-Paper.pdf

  25. Deep Contextualized Word Representations

    https://aclanthology.org/N18-1202.pdf

  26. Is Cosine-Similarity of Embeddings Really About Similarity?

    https://arxiv.org/html/2403.05440

  27. scikit-learn: Nearest Neighbors

    https://scikit-learn.org/stable/modules/neighbors.html

  28. Retrieval Augmented Generation in the Wild

    https://www.youtube.com/watch?v=FDEmbYPgG-s&t=541

  29. Hubs in Space: Popular Nearest Neighbors in High-Dimensional Data

    https://jmlr.org/papers/volume11/radovanovic10a/radovanovic10a.pdf

  30. How to Use t-SNE Effectively

    https://distill.pub/2016/misread-tsne/

  31. UMAP: Using UMAP for Clustering

    https://umap-learn.readthedocs.io/en/latest/clustering.html

  32. On Linear Identifiability of Learned Representations

    https://proceedings.mlr.press/v139/roeder21a.html

  33. Masked Autoencoders Are Scalable Vision Learners

    https://arxiv.org/html/2111.06377

  34. Improving Distributional Similarity with Lessons Learned from Word Embeddings

    https://aclanthology.org/Q15-1016.pdf

  35. A Simple Framework for Contrastive Learning of Visual Representations

    https://proceedings.mlr.press/v119/chen20j/chen20j.pdf

  36. Representation Learning with Contrastive Predictive Coding

    https://arxiv.org/html/1807.03748

  37. Sentence Transformers: Losses

    https://www.sbert.net/docs/package_reference/sentence_transformer/losses.html

  38. Debiased Contrastive Learning

    https://proceedings.neurips.cc/paper/2020/file/63c3ddcc7b23daa1e42dc41f9a44a873-Paper.pdf

  39. Foundations of Computer Vision: Training for Robustness and Generality

    https://visionbook.mit.edu/data_augmentation.html

  40. Foundations of Computer Vision: Convolutional Neural Nets

    https://visionbook.mit.edu/convolutional_neural_nets.html

  41. VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning

    https://arxiv.org/html/2105.04906

  42. How Transformers Finally Ate Vision

    https://www.youtube.com/watch?v=VhfAVA3BG2I&t=523

  43. Adversarial Removal of Demographic Attributes from Text Data

    https://aclanthology.org/D18-1002.pdf

  44. The Hidden Life of Embeddings

    https://www.youtube.com/watch?v=YvobVu1l7GI&t=916

  45. The Hidden Life of Embeddings

    https://www.youtube.com/watch?v=YvobVu1l7GI&t=952

  46. Text Embeddings Reveal (Almost) As Much As Text

    https://aclanthology.org/2023.emnlp-main.765.pdf

  47. MTEB: Massive Text Embedding Benchmark

    https://aclanthology.org/2023.eacl-main.148.pdf

  48. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models

    https://arxiv.org/html/2104.08663

  49. SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation

    https://aclanthology.org/J15-4004.pdf

  50. Benchmarking semantic code retrieval on Claude Code

    https://www.youtube.com/watch?v=zKk7sDMGDEQ&t=412

  51. The RAG Stack We Landed On After 37 Fails

    https://www.youtube.com/watch?v=2CXn-CByNoo&t=760

  52. How to look at your data; what to look for, how to measure

    https://www.youtube.com/watch?v=jryZvCuA0Uc&t=293

  53. How to look at your data; what to look for, how to measure

    https://www.youtube.com/watch?v=jryZvCuA0Uc&t=322

  54. How to look at your data; what to look for, how to measure

    https://www.youtube.com/watch?v=jryZvCuA0Uc&t=371

  55. Benchmarking semantic code retrieval on Claude Code

    https://www.youtube.com/watch?v=zKk7sDMGDEQ&t=354

  56. Dense Passage Retrieval for Open-Domain Question Answering

    https://arxiv.org/html/2004.04906v3

  57. Dense Passage Retrieval for Open-Domain Question Answering

    https://arxiv.org/html/2004.04906v3

  58. Sentence Transformers: semantic search

    https://www.sbert.net/examples/sentence_transformer/applications/semantic-search/README.html

  59. What We Learned from Using LLMs in Pinterest

    https://www.youtube.com/watch?v=XdAWgO11zuk&t=552

  60. The RAG Stack We Landed On After 37 Fails

    https://www.youtube.com/watch?v=2CXn-CByNoo&t=633

  61. Content-based Filtering

    https://developers.google.com/machine-learning/recommendation/content-based/basics

  62. One model to rule recommendations: Netflix's Big Bet

    https://www.youtube.com/watch?v=AbZ4IYGbfpQ&t=512

  63. Knowledge Graphs & GraphRAG: Techniques for Building Effective GenAI Applications: Zach Blumenthal

    https://www.youtube.com/watch?v=2DyHW23L6Cs&t=1674

  64. Recsys Keynote: Improving Recommendation Systems & Search in the Age of LLMs

    https://www.youtube.com/watch?v=2vlCqD6igVA&t=1086

  65. Two-tower retrieval, approximation and compatible deployment artifacts

    https://www.tensorflow.org/recommenders/examples/efficient_serving

  66. 120k players in a week: Lessons from the first viral CLIP app: Joseph Nelson

    https://www.youtube.com/watch?v=OimPoLxioYg&t=55

  67. Zero-Shot Content Moderation with OpenAI's New CLIP Model

    https://blog.roboflow.com/zero-shot-content-moderation-openai-new-clip-model/

  68. When and Why Vision-Language Models Behave like Bags-of-Words, and What to Do About It?

    https://arxiv.org/html/2210.01936v3

  69. Building Multimodal AI Agents (From Scratch)

    https://www.youtube.com/watch?v=640KMYtxCeI&t=1911

  70. Word Embedding-based Antonym Detection Using Thesauri and Distributional Information

    https://aclanthology.org/N15-1100.pdf

  71. The PASCAL Recognising Textual Entailment Challenge

    https://www.cis.upenn.edu/~danroth/Teaching/CS598-05/Papers/rte05_proceedings.pdf

  72. Graph Intelligence: Enhance Reasoning and Retrieval Using Graph Analytics

    https://www.youtube.com/watch?v=GGxAQVbwBL4&t=778

  73. Stop Using RAG as Memory

    https://www.youtube.com/watch?v=T5IMo5ntyhA&t=165

  74. PROV-DM: The PROV Data Model

    https://www.w3.org/TR/prov-dm/

  75. Migrate to a New Embedding Model with Zero Downtime in Qdrant

    https://qdrant.tech/documentation/tutorials-operations/embedding-model-migration/

  76. Elasticsearch aliases

    https://www.elastic.co/docs/manage-data/data-store/aliases

  77. How LLMs work for Web Devs: GPT in 600 lines of Vanilla JS

    https://www.youtube.com/watch?v=ZuiJjkbX0Og&t=2364

  78. How LLMs work for Web Devs: GPT in 600 lines of Vanilla JS

    https://www.youtube.com/watch?v=ZuiJjkbX0Og&t=4748