DataAdvanced10h

Reranking & hybrid search

Retrieve broadly, then rank precisely.

Vector search is fast and rough: it finds passages about roughly the right subject, and often puts mediocre ones near the top. Reranking is a second pass over its output — take the fifty candidates it returned and score each one properly with a slower model that reads the question and the passage together, then keep the best five. Hybrid search is the companion idea: run an old-fashioned keyword search alongside the vector one and merge both lists, because keyword search finds exact strings like product codes and surnames that embeddings blur away.

Embeddings match subject matter, not relevance, so a passage about the right topic that does not answer the question will happily come back first. If the wrong five passages reach the model, the answer is wrong no matter how good the model is, and it will be wrong confidently and with citations.

It sits between the vector database and RAG, and it is usually the cheapest large improvement available to a retrieval system that already works. The pattern to remember is recall first, precision second: cast wide with the cheap search, then spend real compute on the shortlist. It pairs with evaluation, because you cannot tell whether a reranker helped without measuring retrieval quality apart from answer quality.

Retrieve broadly with the cheap method, then spend real compute ranking the shortlist.

In more detailVector search alone misses exact terms, product codes, names, rare words, which keyword search finds trivially. Running both and fusing the results, then reranking the shortlist with a slower, more accurate model, is the standard high-quality pattern. Recall first, precision second.

Learn first

Retrieval-augmented generation

Section 1 of 2, 6h

Retrieve broadly, then rank precisely

By the end: You can substantially improve a mediocre retrieval system.

Why one stage is not enough
Hybrid search and rank fusion
Cross-encoder reranking
Measuring the improvement

One retriever has to do two jobs that pull against each other: find everything relevant, and put the best of it first. A method fast enough to scan a million documents cannot afford to read the query and a document together, so it compares two independently-made vectors and is approximate by construction. The fix is to split the jobs. Stage one fetches fifty candidates and is judged only on whether the right answer is somewhere in them; stage two reads each candidate against the query properly and reorders. This section builds both halves, the fusion that joins two stage-one retrievers, and the arithmetic that decides how many candidates you can afford.

Why one stage is not enough#

The two stages are measured by different numbers, and this is the part that is worth getting straight before any code. Stage one is judged on recall at k: of the documents that should have been found, how many are in the fifty I fetched? Getting them in a good order is not its job. Stage two is judged on precision at the top: given those fifty, are the best three now first? Optimising stage one for ordering is wasted effort, and measuring stage two on recall cannot show anything, since it can only reorder what it was given. Tuning the wrong number for the wrong stage is the most common way this architecture is got wrong.

Stage two cannot fix stage one

If the answer is not in the fifty candidates, no reranker recovers it — it never sees the document. So when quality is poor, measure recall at fifty first. If that is 0.95, the problem is ordering and a reranker will help; if it is 0.60, a reranker is the wrong purchase and the retrieval or the chunking is what needs work. This one measurement decides which half of the system to spend a week on.

Hybrid search: the lexical half#

Embeddings are weak at exactly the things users type most literally: error codes, product names, identifiers, rare acronyms. A vector for "429" is not distinctive, because the model never had a reason to make it so. Keyword search is excellent at precisely those and hopeless at paraphrase, so production retrieval usually runs both. The keyword half is BM25, which is short enough to write out, and worth writing out once because the two constants in it are the whole idea.

import math, re
from collections import Counter

def tokenize(text):
    return re.findall(r"[a-z0-9]+", text.lower())

class BM25:
    def __init__(self, docs, k1=1.5, b=0.75):
        self.k1, self.b = k1, b                  # k1: saturation.  b: length penalty
        self.docs = [tokenize(d) for d in docs]
        self.lengths = [len(d) for d in self.docs]
        self.avg_len = sum(self.lengths) / len(self.docs)
        self.freqs = [Counter(d) for d in self.docs]
        df = Counter(t for d in self.docs for t in set(d))
        n = len(self.docs)
        # Rare terms count for more. The halves keep it finite for a term in every doc.
        self.idf = {t: math.log((n - c + 0.5) / (c + 0.5) + 1) for t, c in df.items()}

    def score(self, query, i):
        total = 0.0
        for t in tokenize(query):
            if t not in self.freqs[i]:
                continue
            f = self.freqs[i][t]
            norm = 1 - self.b + self.b * self.lengths[i] / self.avg_len
            total += self.idf[t] * f * (self.k1 + 1) / (f + self.k1 * norm)
        return total

    def top(self, query, k=3):
        order = sorted(range(len(self.docs)), key=lambda i: -self.score(query, i))
        return [(i, round(self.score(query, i), 3)) for i in order[:k]]

docs = [
    "The API rate limit is 100 requests per minute per API key",
    "Throttling behaviour and what a 429 response means for clients",
    "Reset your password from the account settings page",
    "Billing runs monthly and invoices are issued on the first",
    "A 429 is returned when you exceed the rate limit",
]
bm = BM25(docs)
print(bm.top("429 rate limit"))
print("idf 429:", round(bm.idf["429"], 3), " idf the:", round(bm.idf["the"], 3))

# [(4, 2.626), (0, 1.606), (1, 0.875)]
# idf 429: 0.875  idf the: 0.288

Document 4 wins because it has both terms. The idf line is the mechanism: 429 appears in two of five documents and counts for three times what 'the' does. An embedding has no equivalent lever.

The two constants are worth understanding rather than copying. k1 controls saturation: a document mentioning your term ten times is more relevant than one mentioning it twice, but not five times more, so the score flattens. b controls length: without it, long documents win by accident because they contain more of everything. Defaults of 1.5 and 0.75 are used almost universally and are rarely the thing worth tuning.

Fusing two rankings#

Now you have two ranked lists whose scores mean different things — BM25 is unbounded and starts at zero, cosine sits between minus one and one. Adding them lets whichever has the larger range decide everything, and normalising them per query is fragile because the ranges move with the query. Reciprocal rank fusion sidesteps this by throwing the scores away and using only the positions.

def rrf(rankings, k=60):
    """Each list votes by position. Scores are never compared."""
    scores = {}
    for ranking in rankings:
        for pos, doc in enumerate(ranking):
            scores[doc] = scores.get(doc, 0.0) + 1.0 / (k + pos + 1)
    return sorted(scores, key=scores.get, reverse=True)

lexical  = ["d4", "d1", "d7", "d2"]      # BM25 order
semantic = ["d7", "d4", "d9", "d1"]      # embedding order
print(rrf([lexical, semantic]))

# ['d4', 'd7', 'd1', 'd9', 'd2']
#   d4: lex 1st (0.01639) + sem 2nd (0.01613)   <- in both, so it rises
#   d7: lex 3rd (0.01587) + sem 1st (0.01639)
#   d9: sem 3rd only (0.01587)                  <- in one, so it sinks

Agreement is what the formula rewards: anything both retrievers liked outranks anything only one of them found. The constant 60 comes from the original paper and flattens the difference between first and second place so a single confident retriever cannot dominate; it is not a tuned parameter and moving it rarely helps.

Cross-encoder reranking#

Stage two can afford what stage one cannot: putting the query and the document into the model together, so that attention runs across both and the score reflects them jointly rather than comparing two vectors made in ignorance of each other. That is a cross-encoder, and it is more accurate for the same reason it is slower — nothing can be precomputed, because the pair does not exist until the query arrives. A bi-encoder embeds a million documents once; a cross-encoder must do one forward pass per candidate, per query.

bi-encoder  (stage one)          cross-encoder (stage two)
  embed(query)  -> vector           model(query, document) -> score
  embed(doc)    -> vector           both texts, one pass, attention across
  cosine between them               nothing precomputable
  documents embedded once           one pass per candidate per query
  millions of docs, milliseconds    tens of candidates, tens of ms each

This is the whole tradeoff, and it is why the architecture has two stages rather than one good one.

Which makes the candidate count a latency decision rather than a quality one. At a rough 8 ms per pair on a small cross-encoder, reranking 10 candidates adds 80 ms, 50 adds 400 ms and 100 adds 800 ms — to every query, on top of retrieval and generation. Measure the per-pair cost of your own model and multiply, because that product is what sets your candidate count. Most systems land between 20 and 50, and the way to choose is to measure recall at k on your own data and take the smallest k where it stops climbing.

Measuring the improvement#

The honest test is a set of queries with known correct documents — a hundred is enough to see a real effect — scored before and after the reranker, with the stages measured separately.

stage one, recall@50      is the answer in the candidates at all?
stage two, MRR            1/rank of the first correct document, averaged
stage two, precision@3    of the three you will actually show, how many are right
end to end                answer quality, which is the only one that pays

Report the first two separately. A reranker that improves MRR while recall@50 stays flat is doing its job; one that changes neither is not earning its latency and should be removed.

And measure the latency in the same run. A reranker is a permanent tax on every query, so the comparison that decides whether it ships is the quality gain against the added milliseconds and the added cost — not the quality gain alone. If the gain is small and the budget is tight, better chunking and a query rewrite are usually the cheaper purchase, and both are covered in the sections around this one.

Where people go wrong

Adding reranking to fix bad recall. If the answer never made the shortlist, reordering the shortlist changes nothing. Diagnose which stage is failing first.

Build this: Build a retrieval eval over thirty questions with known correct chunks. Measure recall at 5 with vector search alone, then with hybrid, then with reranking, and record the latency cost of each step.

Open this section on its own page

Section 2 of 2, 4h

Fixing the query, not just the ranking

By the end: You can improve retrieval from the query side, and measure each technique rather than stacking them.

Turning a chat turn into a standalone question
Several phrasings of one query
Hypothetical answers as queries
Breaking a question into parts
Pulling filters out of the question
Metrics: recall@k, MRR

Retrieval failures often have nothing to do with the index. The user typed 'and what about for teams?', and no document resembles that. This section is about the step before the search: rewriting a conversational turn into a question that stands on its own, and the other query transformations that sometimes help, such as several phrasings, a hypothetical answer, or splitting a question in two. Each costs a model call and some latency, so the section also covers the measurements that tell you which ones to keep.

The query is often the problem#

In a conversation, the user's latest message is often 'and what about for teams?'. Embedding that finds nothing useful. The question only makes sense with the turns before it.

Rewrite into a standalone question#

History:  "How do refunds work on the monthly plan?"
          "Within 14 days, in full."
Latest:   "and for teams?"

Rewritten: "How do refunds work on the monthly team plan?"

One cheap model call before retrieval. For chat interfaces this is usually the single biggest improvement available.

Other techniques, each with a cost#

Several phrasings: generate three versions of the query, search with each, fuse the results by rank. Helps when users phrase things unlike the documents.
Hypothetical answer: ask a model to write a plausible answer, and search with that. Answers look more like documents than questions do. Risky when the model's guess points the search in the wrong direction.
Decomposition: split 'compare the refund policy of plan A and plan B' into two searches.
Filters: extract structured conditions such as a date range or product name, and apply them as metadata filters rather than hoping the embedding captures them.

A corpus small enough to argue with#

The rest of this section is measured rather than asserted, on fourteen chunks of a fictional pricing and policy document and five real-shaped follow-up turns. The retriever is a character n-gram TF-IDF index, which stands in for an embedding model so the run needs no network and gives the same numbers every time. It is kinder than a word-level index and less capable than a real embedding model, and the direction of every result below survives that substitution even though the exact figures would not.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

CHUNKS = {
    "c1": ("Refunds on the monthly plan are issued in full within 14 days of purchase.", "monthly"),
    "c2": ("Refunds on the annual plan are pro-rated after the first 30 days.", "annual"),
    "c3": ("Team plans are billed per seat and refunds follow the monthly policy for each seat.", "team"),
    "c4": ("Adding a seat to a team plan is charged pro-rata for the rest of the billing period.", "team"),
    "c5": ("To reset your password, use the Forgot password link on the sign-in page.", "any"),
    "c6": ("The free tier allows 60 requests per minute.", "free"),
    "c7": ("The paid tier allows 6000 requests per minute and bursts to 12000.", "paid"),
    "c8": ("Invoices are issued on the first of the month and sent to the billing contact.", "any"),
    "c9": ("Single sign-on is available on team and enterprise plans.", "team"),
    "c10": ("Cancelling a subscription stops renewal at the end of the current period.", "any"),
    "c11": ("Data is deleted 30 days after an account is closed.", "any"),
    "c12": ("Enterprise plans include a dedicated support channel and an uptime commitment.", "enterprise"),
    "c13": ("The monthly plan costs 20 per user and is billed on the day you signed up.", "monthly"),
    "c14": ("Support responses are within one business day on paid plans.", "paid"),
}

IDS = list(CHUNKS)
TEXTS = [CHUNKS[i][0] for i in IDS]
VEC = TfidfVectorizer(analyzer="char_wb", ngram_range=(3, 5)).fit(TEXTS)
M = VEC.transform(TEXTS)

def search(query, k=3, allowed=None):
    sims = cosine_similarity(VEC.transform([query]), M)[0]
    pool = [(i, s) for i, s in zip(IDS, sims) if allowed is None or i in allowed]
    pool.sort(key=lambda p: -p[1])
    return [i for i, _ in pool[:k]]

print("raw       ", search("what about SSO?"))
print("rewritten ", search("Is single sign-on available on team plans?"))

# raw        ['c4', 'c3', 'c10']
# rewritten  ['c9', 'c13', 'c4']

The raw turn returns three chunks about seats, teams and cancellation, and the one chunk that mentions single sign-on is not among them. Nothing is broken: the index was asked about SSO and there is no chunk about SSO in the abstract, only one about SSO on team plans.

Measure every step#

Each technique adds latency and a model call. Some help your data and some do not. Build a set of real questions with the chunks that answer them, and measure after each change.

def recall_at_k(results, relevant, k=5):
    return len(set(results[:k]) & relevant) / len(relevant)

def reciprocal_rank(results, relevant):
    for i, r in enumerate(results, start=1):
        if r in relevant:
            return 1 / i
    return 0.0   # average this over queries to get MRR

Two numbers, and they answer different questions. Recall at k asks whether the answering chunk reached the shortlist at all, which is the only thing that decides whether the model can possibly be right. MRR asks how far up it landed, which decides whether it survives a context budget that keeps three chunks. A change can move one and not the other, and knowing which it moved tells you whether to keep it.

QUERIES = [
    {"raw": "and for teams?",
     "rewritten": "How do refunds work on the team plan?",
     "relevant": {"c3"}},
    {"raw": "and on the paid one?",
     "rewritten": "What is the rate limit on the paid tier?",
     "relevant": {"c7"}},
    {"raw": "what about SSO?",
     "rewritten": "Is single sign-on available on team plans?",
     "relevant": {"c9"}},
    {"raw": "and if I add one mid-month?",
     "rewritten": "What happens if I add a seat to a team plan mid-month?",
     "relevant": {"c4"}},
    {"raw": "how long do you keep it?",
     "rewritten": "How long is data kept after an account is closed?",
     "relevant": {"c11"}},
]

def evaluate(get_results, label, calls):
    """Every variant returns ten candidates, so the columns are comparable."""
    rec = sum(recall_at_k(get_results(q), q["relevant"], k=3) for q in QUERIES) / len(QUERIES)
    mrr = sum(reciprocal_rank(get_results(q), q["relevant"]) for q in QUERIES) / len(QUERIES)
    print(f"{label:<28}{rec:>9.2f}{mrr:>7.2f}{calls:>8}")
    return rec, mrr

print(f"{'query sent to the index':<28}{'recall@3':>9}{'MRR':>7}{'calls':>8}")
evaluate(lambda q: search(q["raw"], k=10), "the raw turn", 0)
evaluate(lambda q: search(q["rewritten"], k=10), "rewritten to standalone", 1)

# query sent to the index      recall@3    MRR   calls
# the raw turn                     0.40   0.35       0
# rewritten to standalone          1.00   1.00       1

The rewrites are hard-coded here, standing in for one cheap model call per turn, so the measurement is about what reaches the index rather than about any particular model. Recall goes from 0.40 to 1.00 and the index was never touched.

Several phrasings, fused#

The idea is that users phrase things unlike the documents do, so generating two or three alternative phrasings and fusing the rankings covers more of the space. Fusing by rank rather than by score is what makes the merge meaningful, and it is the same reciprocal rank fusion the two-stage retrieval section uses to join a keyword and a vector search.

PHRASINGS = {
    "How do refunds work on the team plan?": [
        "team plan refund policy",
        "can a team plan be refunded per seat",
    ],
    "What is the rate limit on the paid tier?": [
        "paid tier requests per minute",
        "how many requests can a paid account make",
    ],
    "Is single sign-on available on team plans?": [
        "SSO support for teams",
        "does the team plan have single sign-on",
    ],
    "What happens if I add a seat to a team plan mid-month?": [
        "adding a seat part way through a billing period",
        "pro-rata charge for a new seat",
    ],
    "How long is data kept after an account is closed?": [
        "data retention after account closure",
        "when is data deleted",
    ],
}

def rrf(rankings, k=60):
    scores = {}
    for ranking in rankings:
        for pos, doc in enumerate(ranking):
            scores[doc] = scores.get(doc, 0.0) + 1.0 / (k + pos + 1)
    return sorted(scores, key=scores.get, reverse=True)

def multi(q):
    variants = [q["rewritten"]] + PHRASINGS[q["rewritten"]]
    return rrf([search(v, k=5) for v in variants])[:10]

evaluate(multi, "three phrasings, fused", 2)

# three phrasings, fused           1.00   1.00       2

Identical to the row above it, for a second model call and three searches instead of one. That is the result to expect once the rewrite has already done its work, and it is why this technique is measured rather than assumed: it earns its place on corpora where the vocabulary genuinely differs from the users', and nowhere else.

Searching with a hypothetical answer#

A question and a passage that answers it do not look alike. Ask a model to write the answer it would expect, and search with that instead, and you are comparing a passage to a passage. It works when the model's guess is in the right neighbourhood and misfires when it is not, and the misfire is worth seeing rather than reading about.

HYDE = {
    "How do refunds work on the team plan?":
        "Refunds on team plans are pro-rated and issued within 30 days of the request.",
    "What is the rate limit on the paid tier?":
        "The paid tier allows several thousand requests per minute.",
    "Is single sign-on available on team plans?":
        "Single sign-on is available on team plans and can be configured in settings.",
    "What happens if I add a seat to a team plan mid-month?":
        "A seat added mid-month is charged pro-rata for the remainder of the period.",
    "How long is data kept after an account is closed?":
        "Data is retained for a period after closure and then permanently deleted.",
}

evaluate(lambda q: search(HYDE[q["rewritten"]], k=10), "hypothetical answer", 2)
for q in QUERIES[:1]:
    print("  hyde query:", HYDE[q["rewritten"]])
    print("  returns   :", search(HYDE[q["rewritten"]]), "wanted", q["relevant"])

# hypothetical answer              1.00   0.77       2
#   hyde query: Refunds on team plans are pro-rated and issued within 30 days of the request.
#   returns   : ['c1', 'c2', 'c3'] wanted {'c3'}

Same recall, worse MRR, and the last two lines say why. The model guessed that team refunds are pro-rated within 30 days; the document says they follow the monthly policy. The invented phrase pulled the monthly and annual refund chunks above the team one, so the right chunk survives at rank three instead of rank one. On a pipeline that keeps two chunks it would have been lost.

Pulling the filter out of the question#

Some of what a question says is not a topic at all but a condition: a plan name, a product, a region, a date range. An embedding has to spend similarity on those words along with everything else, and it is bad at them, because one plan name looks much like another. Extract them and apply them as a metadata filter, and the search runs over the rows that could be right rather than over everything.

import re

PLANS = ["monthly", "annual", "team", "free", "paid", "enterprise"]

def extract_filter(question):
    found = [p for p in PLANS if re.search(rf"\b{p}\b", question.lower())]
    return found[0] if len(found) == 1 else None

question = "Is a refund issued in full on the annual plan?"
plan = extract_filter(question)
allowed = {i for i, (_, p) in CHUNKS.items() if p == plan}
print("filter:", plan, "candidates:", len(allowed), "of", len(CHUNKS))
print("no filter:", search(question))
print("filtered :", search(question, allowed=allowed))

# filter: annual candidates: 1 of 14
# no filter: ['c1', 'c2', 'c8']
# filtered : ['c2']

Without the filter the top result is the monthly refund policy, because the question borrowed the words 'in full' from it. That is the answer a customer on the annual plan would have been given, and no reranker would have caught it, because c1 genuinely is the closest text.

Two rules keep this from causing its own outage. Extract a filter only when it is unambiguous — the code above returns nothing when two plan names appear, because filtering on the wrong one is worse than not filtering. And never let an extracted filter empty the result set silently: if the filtered search returns nothing, fall back to the unfiltered one and say so in the trace, or a typo in a product name turns into a confident 'I could not find anything'.

Breaking a question into parts#

A question with two subjects has no good single embedding, because the vector lands between them and is close to neither. Splitting it, searching each part and fusing is the fix, and it is worth doing only for the questions that need it — a classifier or a short model call decides, and most turns are not compound.

compound = "compare refunds on the monthly plan with the rate limit on the free tier"
parts = ["refunds on the monthly plan", "the rate limit on the free tier"]
want = {"c1", "c6"}
print("one search :", search(compound), "recall", recall_at_k(search(compound), want))
merged = rrf([search(p, k=3) for p in parts])
print("decomposed :", merged[:3], "recall", recall_at_k(merged, want))

# one search : ['c1', 'c2', 'c13'] recall 0.5
# decomposed : ['c2', 'c1', 'c6'] recall 1.0

The single search found the refund half and filled the rest of the shortlist with more refund chunks, so half the question was unanswerable before generation started. The decomposed version costs two searches and one extra model call, and is the only one of these techniques whose benefit is structural rather than incidental.

Put the four rows together and the shape of the decision is clear. The standalone rewrite is the one that pays on a chat interface, and it pays enormously. Multiple phrasings cost a call and returned nothing here. A hypothetical answer kept recall and damaged ranking. Decomposition is worth having for the questions that need it and nothing for the ones that do not. The measurement is cheap — fourteen chunks and five questions took an afternoon to write — and it is the only thing that distinguishes those four outcomes, which is why stacking all of them from a blog post leaves you slower and unable to say what helped.

Do not rewrite what is already standalone

The rewrite adds a model call and a few hundred milliseconds to every turn, including the first one, which by definition has no history to fold in. A cheap test skips most of that: if the turn is the first in the conversation, or it is long enough and contains a noun the history does not supply, send it as it is. Keep the check conservative, because a turn wrongly judged standalone is retrieved on the wrong query and the user sees a confidently irrelevant answer, while a turn wrongly rewritten costs only the call.

Where people go wrong

Adding every query technique from a blog post at once. Latency triples and you cannot tell which one helped.

Build this: Collect twenty real follow-up questions from a chat log with their answering chunks. Measure recall@5 and MRR for raw messages, then with standalone rewriting, then with one more technique of your choice. Keep only what earns its latency.

Open this section on its own page

Where you use this

A topic is easier to care about when you can see what it is for. These projects lean on it at a named stage.

RAG with citations you can check

Stage 5, Retrieve widely, then rank precisely: Why a cheap wide first pass and an expensive narrow second pass beats one good retriever.

Now actually do it

Reading a topic is not learning it. These are the hours that count, and each one tells you when you are finished.

1Measure each retrieval stage

Build a retrieval eval over 30 questions with known correct chunks. Measure recall@5 for vector only, then hybrid, then with reranking, recording latency at each step.

Done when: A table of recall and latency for all three, which is what justifies the added complexity.

6 hours of practice

Words used here

5 terms this topic teaches, each with a one-line definition if one of them stopped you.

Reranking: A second, more expensive scoring pass over the top few dozen retrieved candidates, reordering them before the best handful go into the prompt.
Cross-encoder: A model that reads query and document together in one pass and outputs a relevance score. Much more accurate than comparing separate embeddings, and far too slow to run over a whole corpus — hence its use as a reranker.
Bi-encoder: A model encoding query and document separately, so document vectors can be computed once and stored. This independence is what makes first-stage retrieval over millions of documents possible.
Hybrid search: Running keyword and vector retrieval together and merging the rankings. Each catches what the other misses: exact identifiers, names and rare words on one side, paraphrase and synonymy on the other.
BM25: The standard keyword ranking function, scoring a document by how often the query's rare terms appear in it, adjusted for document length.

Interview questions

7 questions, grouped by the section that teaches the answer. An answer you have read is not an answer you can give, so use Test yourself and say it out loud first.

Retrieve broadly, then rank precisely

Why use two retrieval stages?

Recall and precision want different things. The first stage is cheap and broad so the answer is definitely in the candidate set. The second is expensive and accurate and orders a short list. One stage has to compromise between them.

Why fuse search results by rank rather than score?

A BM25 score and a cosine similarity are on different scales with no meaningful conversion, so combining raw scores is arbitrary. Reciprocal rank fusion uses only position, which is comparable across any two ranked lists.

Then they push further

How does reciprocal rank fusion work?

Each document scores one divided by a constant plus its rank, summed over every list it appears in. A document ranked well by both searches wins. The constant, commonly 60, stops a single first place from dominating.

What gets marked up. Strong answers explain that keyword and vector scores are on different scales and cannot be compared directly, which is the whole reason to use ranks.

What is a cross-encoder and why not use it for everything?

It reads query and candidate together and scores the pair, which is much more accurate than comparing independently computed vectors. It also means one forward pass per candidate, so it is far too slow to run over a whole corpus.

Fixing the query, not just the ranking

Your RAG chatbot answers the first question well and follow-ups badly. Why, and what do you do?

Follow-ups depend on earlier turns, so embedding the raw message retrieves the wrong things. Rewrite each turn into a standalone question using the history before retrieving, and measure the change on a set of real follow-up questions.

Then they push further

The rewrite adds 300ms. Is it worth it?

Measure it. If recall on follow-ups rises substantially, usually yes, and a small fast model keeps the delay down. Skip the rewrite when the message is already self-contained, which a cheap check can detect.

What gets marked up. Marked up for spotting that the query lacks context and for measuring the fix rather than stacking techniques.

What is MRR and when would you use it rather than recall@k?

Mean reciprocal rank averages one over the position of the first relevant result. It rewards getting a relevant result to the very top, which matters when only the first few results are used. Recall@k measures whether the relevant results made it into the set at all.

When would you search with a hypothetical answer rather than the question, and what is the risk?

When questions and passages are written differently enough that a question embedding sits far from the passage that answers it. You ask a model for the answer it would expect and search with that, so you are comparing a passage to a passage. The risk is that the guess is specific and wrong: an invented detail becomes part of the query and pulls the search towards documents containing that detail rather than the correct one. Measure recall and MRR separately, because the usual damage is to the ranking rather than to whether the right chunk is retrieved at all.

Then they push further

How would you make it safer?

Search with the question and the hypothetical answer and fuse the rankings, so an off-target guess cannot dominate. Keep the hypothetical short and generic. And measure on questions where your corpus disagrees with common knowledge, since that is where the model's guess is confidently wrong.

What gets marked up. Marked up for naming the failure concretely, and for knowing the technique trades ranking for vocabulary match rather than being a free improvement.

How do you decide which query techniques to keep?

Build a small labelled set: twenty or so real turns from a chat log with the chunks that answer them. Measure recall@k and MRR for each technique added one at a time, and record the extra model calls and latency beside them. Keep the ones that move a number you care about. On most chat corpora the standalone rewrite moves recall a long way and the rest move it very little, and you cannot tell which case you are in without the measurement.

These questions on their own page, one link per question