DataCore5h

Embeddings

Text as coordinates, so similarity becomes distance.

An embedding turns a piece of text into a long list of numbers — a few hundred or a few thousand of them. Think of those numbers as coordinates, the way latitude and longitude place a town on a map. Text about similar things lands in a similar place, so "how do I reset my password" ends up near "I forgot my login" even though they share almost no words.

Computers cannot compare meaning, but they can compare numbers very quickly. Once text is coordinates, finding the most relevant document out of a million becomes arithmetic — measuring which points are closest — instead of a search for matching words that would miss every paraphrase.

It is the step that makes retrieval possible: chunking cuts documents up, embeddings place each piece on the map, a vector database stores the map and searches it, and RAG uses what comes back. Learn this before any of those and they all make sense.

Similar meaning becomes nearby coordinates, which turns search into measuring distance.

In more detailAn embedding maps text to a vector such that related meanings land near each other, turning retrieval into nearest-neighbour search. The practical skill is knowing the failure mode: embeddings capture topical similarity, not logical relevance, so they cheerfully return passages about the right subject that do not answer the question.

Learn first

Tokenization

Section 1 of 2, 2h

Turning meaning into coordinates

By the end: You can explain what an embedding is and use one to find similar text.

What an embedding actually is
Why similar meaning lands nearby
Cosine similarity
Making your first search work

This is the mechanism behind searching by meaning: representing a piece of text as a position in space, so that finding related material becomes a matter of finding nearby points. It is what lets a question about refunds surface the right passage when that passage never uses the word. The section covers what those positions are, how nearness between them is measured, and how to get a first search working over text of your own.

A list of numbers that means something#

An embedding model takes text and returns a list of numbers, typically a few hundred to a few thousand of them. That list is a position in space. Text with similar meaning gets a position nearby.

"dog"    ->  [0.21, -0.43, 0.88, ...]
"puppy"  ->  [0.19, -0.41, 0.85, ...]     very close to dog
"invoice"->  [-0.62, 0.11, -0.30, ...]    far away

The individual numbers mean nothing you can read. Only the distances matter. That is enough to turn 'find me something similar' into arithmetic.

Measuring closeness#

Cosine similarity measures the angle between two vectors, ignoring their length. It returns 1 for identical direction, 0 for unrelated and -1 for opposite. It is the standard choice because it cares about meaning rather than magnitude.

import numpy as np

def cosine(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

cosine(embed("dog"), embed("puppy"))    # about 0.9
cosine(embed("dog"), embed("invoice"))  # about 0.1

A search in twenty lines#

docs = ["Refunds take 5 working days.",
        "Our office is in Berlin.",
        "You can return items within 30 days."]

doc_vectors = [embed(d) for d in docs]      # do this once, store it

def search(query, k=2):
    q = embed(query)
    scored = [(cosine(q, v), d) for v, d in zip(doc_vectors, docs)]
    scored.sort(reverse=True)
    return scored[:k]

search("how long for my money back")
# [(0.81, 'Refunds take 5 working days.'), ...]

The query shares no words with the answer. That is the whole point of embedding search.

Embed once, store the result

Embedding is an API call with a cost. Embed your documents once and keep the vectors. Only the query is embedded at search time. Re-embedding a corpus on every search is a common and expensive beginner mistake.

Where people go wrong

Embedding the query and the documents with different models, or with different settings on the same model. The results look plausible and are meaningless.

Build this: Embed ten sentences, pick one as a query, and rank the others by similarity. Then write a query sharing no words with the best match and confirm it still ranks first.

Open this section on its own page

Section 2 of 2, 3h

Where embedding search fails

By the end: You can predict which queries will fail and fix them.

Topical similarity is not relevance
Exact terms that embeddings lose
Hybrid search
Reranking

Vector search is easy to get working and easy to trust too far. This section is about the queries it handles badly, which is a wider set than the demos suggest: exact identifiers, proper names, negations, and anything where the passage you need is worded nothing like the question. You will learn to predict those in advance, and the two standard repairs, running a keyword search alongside the vector search and reranking the combined results.

Similar is not the same as useful#

This is the most important thing to understand about retrieval. Embeddings find text about the same subject. They do not find text that answers your question. Those overlap often enough to be encouraging and diverge often enough to be a problem.

Query:   "Does the warranty cover water damage?"

Retrieved, scoring highly, all about warranties:
  "Our warranty lasts 24 months."
  "Warranty claims require a receipt."
  "The warranty covers manufacturing defects."

None of them answer the question.

Exact terms disappear#

Embeddings compress meaning, and in doing so they lose precision on exact strings. A product code, an error number, a surname, a rare technical term: these are exactly what a user searches for, and exactly what embeddings are worst at.

Query: "error E4471"

Embedding search:  returns general text about errors
Keyword search:    returns the document containing E4471

Use both#

Hybrid search runs a keyword search and a vector search and combines the results. Keyword search catches exact terms, vector search catches paraphrase. Together they cover each other's blind spots, and this is usually the single largest quality improvement available to a mediocre retrieval system.

def hybrid(query, k=10):
    keyword = bm25_search(query, k=k)
    vector = vector_search(query, k=k)

    # Reciprocal rank fusion: score by position, not by raw score,
    # because the two systems' scores are not on the same scale.
    scores = {}
    for rank, doc in enumerate(keyword):
        scores[doc.id] = scores.get(doc.id, 0) + 1 / (60 + rank)
    for rank, doc in enumerate(vector):
        scores[doc.id] = scores.get(doc.id, 0) + 1 / (60 + rank)

    return sorted(scores, key=scores.get, reverse=True)[:k]

Fusing by rank sidesteps the fact that a BM25 score and a cosine score mean different things.

Rerank the shortlist#

A reranker is a slower, more accurate model that scores each candidate against the query directly, rather than comparing two independently computed vectors. It is too slow to run over a whole corpus and ideal for reordering twenty candidates down to five.

Retrieve broadly    50 candidates    fast, high recall
Rerank              top 20           slower, high precision
Send to the model   top 5            expensive per token

Cast wide, then narrow. Each stage is more accurate and more expensive than the last.

Diagnose recall before precision

If the right document was never retrieved, no amount of reranking helps. Check first whether the answer is in your top fifty. If it is, you have a ranking problem. If it is not, you have a chunking or an embedding problem, and they need different fixes.

Where people go wrong

Tuning the prompt when retrieval is the problem. If the right passage never reached the model, no prompt can fix it, and teams lose weeks this way.

Build this: Build a search over fifty documents. Write five queries that embedding search handles well and five it handles badly, including one with an exact product code. Then add keyword search and measure the improvement.

Open this section on its own page

Where you use this

A topic is easier to care about when you can see what it is for. These projects lean on it at a named stage.

RAG with citations you can check

Stage 2, Embed and index: What the numbers mean, and why cosine similarity is the comparison being made.
Stage 3, Read what comes back: The query types embedding search is bad at, so you recognise them in your own results.

Now actually do it

Reading a topic is not learning it. These are the hours that count, and each one tells you when you are finished.

1Search with no shared words

Embed 50 sentences and write five queries that share no vocabulary with their correct answer. Confirm retrieval still works.

Done when: All five retrieve the right passage despite zero word overlap.

2Find where embeddings fail

Write five queries embedding search handles badly, including one with an exact product code. Then add keyword search and measure the improvement.

Done when: You can demonstrate a query that only keyword search finds and explain why.

5 hours of practice

Words used here

2 terms this topic teaches, each with a one-line definition if one of them stopped you.

Embedding model: A model that turns a piece of text into one fixed-length vector capturing its meaning, so similarity can be measured by comparing vectors.
Cosine similarity: The cosine of the angle between two vectors: 1 when they point the same way, 0 when perpendicular. The standard way to score how close two embeddings are, because it ignores magnitude and compares direction alone.

Interview questions

7 questions, grouped by the section that teaches the answer. An answer you have read is not an answer you can give, so use Test yourself and say it out loud first.

Turning meaning into coordinates

What is an embedding?

A vector representing a piece of text, positioned so that text with similar meaning lands nearby. It converts semantic similarity into geometric distance, which makes similarity searchable.

Why cosine similarity rather than Euclidean distance?

Cosine measures the angle and ignores magnitude, so it compares direction of meaning rather than length of text. Euclidean distance would let a long document score differently from a short one saying the same thing.

Can you compare embeddings from two different models?

No. Each model has its own space, so the numbers are not comparable. Changing embedding model means re-embedding the entire corpus, which is why the choice is worth a little thought up front.

Where embedding search fails

Why does embedding search sometimes return irrelevant results?

It measures topical similarity, not whether a passage answers the question. Text about the right subject scores highly even when it contains nothing useful. It also loses exact terms like product codes, which is why hybrid search exists.

What is hybrid search?

Running keyword and vector search together and fusing the results, usually by rank rather than raw score since the two scoring scales are incomparable. Keyword catches exact terms, vector catches paraphrase.

What does a reranker do?

It scores each candidate against the query directly rather than comparing pre-computed vectors, which is more accurate and much slower. You use it to reorder a shortlist, never to search a whole corpus.

Your RAG system returns wrong answers. How do you debug it?

Check recall first: was the correct document in the retrieved set at all? If not, the problem is chunking or embedding. If it was retrieved but ranked low, the problem is ranking, and reranking or hybrid search will help. Only if retrieval was correct is it a generation problem.

Then they push further

The right chunk was retrieved but the answer is still wrong. Now what?

The problem is in generation. Check that the chunk was in the context the model actually saw and not trimmed away, whether it was buried among irrelevant chunks, and whether the prompt tells the model to rely on the context. Then read the answer against the chunk to see if it misread a number or blended two sources.

The right chunk was never retrieved. What next?

Find out why. Try the question as a keyword search to see if the words match at all, check how that document was parsed and chunked, and check whether the question uses different words from the document. Hybrid search, better chunking and query rewriting each fix a different one of these.

How do you stop it coming back?

Add the question to the retrieval eval with the chunk it should find, and to the answer eval with the expected answer. Measure retrieval separately from answer quality so you know which stage regressed.

What gets marked up. Strong answers split the problem into retrieval and generation and check which failed before changing anything. Jumping straight to a new embedding model or a bigger model is marked down.

These questions on their own page, one link per question