Turning meaning into coordinates
By the end: You can explain what an embedding is and use one to find similar text.
- What an embedding actually is
- Why similar meaning lands nearby
- Cosine similarity
- Making your first search work
New to embeddings? An embedding turns a piece of text into a long list of numbers — a few hundred or a few thousand of them. Think of those numbers as coordinates, the way latitude and longitude place a town on a map. Text about similar things lands in a similar place, so "how do I reset my password" ends up near "I forgot my login" even though they share almost no words. Why it matters
This is the mechanism behind searching by meaning: representing a piece of text as a position in space, so that finding related material becomes a matter of finding nearby points. It is what lets a question about refunds surface the right passage when that passage never uses the word. The section covers what those positions are, how nearness between them is measured, and how to get a first search working over text of your own.
A list of numbers that means something#
An embedding model takes text and returns a list of numbers, typically a few hundred to a few thousand of them. That list is a position in space. Text with similar meaning gets a position nearby.
"dog" -> [0.21, -0.43, 0.88, ...]
"puppy" -> [0.19, -0.41, 0.85, ...] very close to dog
"invoice"-> [-0.62, 0.11, -0.30, ...] far awayThe individual numbers mean nothing you can read. Only the distances matter. That is enough to turn 'find me something similar' into arithmetic.
Measuring closeness#
Cosine similarity measures the angle between two vectors, ignoring their length. It returns 1 for identical direction, 0 for unrelated and -1 for opposite. It is the standard choice because it cares about meaning rather than magnitude.
import numpy as np
def cosine(a, b):
return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
cosine(embed("dog"), embed("puppy")) # about 0.9
cosine(embed("dog"), embed("invoice")) # about 0.1A search in twenty lines#
docs = ["Refunds take 5 working days.",
"Our office is in Berlin.",
"You can return items within 30 days."]
doc_vectors = [embed(d) for d in docs] # do this once, store it
def search(query, k=2):
q = embed(query)
scored = [(cosine(q, v), d) for v, d in zip(doc_vectors, docs)]
scored.sort(reverse=True)
return scored[:k]
search("how long for my money back")
# [(0.81, 'Refunds take 5 working days.'), ...]Embed once, store the result
Embedding is an API call with a cost. Embed your documents once and keep the vectors. Only the query is embedded at search time. Re-embedding a corpus on every search is a common and expensive beginner mistake.
Where people go wrong
Embedding the query and the documents with different models, or with different settings on the same model. The results look plausible and are meaningless.
If this comes up in an interview
What is an embedding?
A vector representing a piece of text, positioned so that text with similar meaning lands nearby. It converts semantic similarity into geometric distance, which makes similarity searchable.
Why cosine similarity rather than Euclidean distance?
Cosine measures the angle and ignores magnitude, so it compares direction of meaning rather than length of text. Euclidean distance would let a long document score differently from a short one saying the same thing.
Can you compare embeddings from two different models?
No. Each model has its own space, so the numbers are not comparable. Changing embedding model means re-embedding the entire corpus, which is why the choice is worth a little thought up front.
Build this: Embed ten sentences, pick one as a query, and rank the others by similarity. Then write a query sharing no words with the best match and confirm it still ranks first.
Watch it explained
This is one section of Embeddings, 5 hours across 2 sections. Read it in full to get the whole topic in order.
Stop 21 of 50 on the AI Engineer route
0 of 50 topics done · 0h behind you, 603h to go
What you have marked done is stored in this browser only. It does not reach a server, and clearing site data will clear it.