Retrieval-augmented generation interview questions
8 questions across 2 sections, in the order you would learn them. Every answer here is one that works — but an answer you have read is not an answer you can give, so use Test yourself and say it out loud first.
The RAG pipeline
Read the section that teaches thisIf this comes up in an interview
What problem does RAG solve?
Models only know their training data, so they cannot answer about your private or recent documents, and they tend to invent an answer rather than decline. RAG retrieves the relevant text and puts it in the prompt so the answer comes from evidence.
How would you chunk documents?
On structure rather than a fixed character count: paragraphs and sections, keeping headings attached to their content, with slight overlap between chunks. Store the document title and heading path with each chunk and include them when embedding, so a chunk is self-describing.
Why ask the model to cite sources?
It lets users verify, it lets you debug which chunk caused a wrong answer, and it reduces invention because the model is less willing to assert something it must attribute.
When would you fine-tune instead of using RAG?
Fine-tuning teaches form: a consistent output format, a domain vocabulary, a house style. RAG supplies facts. If the problem is that the model does not know something, RAG is the answer, because fine-tuning does not reliably install new facts and cannot be updated without retraining.
Then they push further
A team wants to fine-tune a model on their docs so it knows the product. Good idea?
Usually not. Fine-tuning is unreliable at adding facts, cannot be updated when a document changes without training again, and cannot cite a source. Retrieval keeps knowledge current and checkable. Fine-tuning fits problems of behaviour, format or style, not knowledge.
Can you use both?
Yes. Retrieve for the facts and fine-tune for how to use them: a house citation style, a strict output format, or making a small model good enough at the task to replace a larger one.
What gets marked up. Strong answers separate knowledge from behaviour and mention keeping content current and citing sources. Weak answers treat fine-tuning as the more serious or advanced option.
Making it actually work
Read the section that teaches thisIf this comes up in an interview
Your RAG system gives a wrong answer. How do you find out why?
Separate the stages. Check whether the correct chunk was retrieved. If it was not, the problem is chunking, embedding or ranking. If it was, feed it deliberately and check the answer, which isolates a prompt or model problem.
How do you handle a question your documents cannot answer?
Threshold on retrieval score so a weak best match does not proceed, instruct the model that declining is acceptable, and include no-answer cases in the eval set. A confident wrong answer costs more trust than a refusal.
Then they push further
How do you measure how well it declines?
Put unanswerable questions in the eval set on purpose and measure how often it answers them anyway, alongside how often it wrongly declines questions it could answer. Improving one usually worsens the other, so track both.
What gets marked up. Marked up for treating 'I do not know' as a correct answer with its own test cases, and for tracking wrong refusals too.
What is query rewriting and why does it matter?
Rewriting the user's question into a standalone search query using the conversation context. Follow-up questions like 'and for international?' are meaningless to a search index on their own, and rewriting often improves retrieval more than index tuning.
How do you keep a RAG index current?
Re-index on change rather than rebuilding, keyed by content hash. Delete chunks for removed documents. Store and display a last-updated date, because a stale answer is indistinguishable from a correct one to the user.
These are the questions for one topic. The whole question bank covers every topic on the site.