Prompting, RAG, or fine-tuning
The output is not good enough. Which lever do you pull, and in what order?
The short answer
Prompt, then retrieve, then tune — in that order, with an eval set built before the first change so you can tell which of them did anything.
Better prompting
Good at
- Being free and immediate — you can try ten versions in an hour.
- Fixing the very common case where the instruction was ambiguous rather than the model incapable.
- Establishing the ceiling: what the model can do when asked properly.
- Carrying format, tone and edge-case rules with a handful of examples.
Costs you
- Long prompts cost tokens and latency on every single call, forever.
- Fragile across model versions — a prompt tuned hard to one model often degrades on the next.
- Improvements are easy to imagine and hard to confirm without an eval set.
- Cannot supply knowledge the model does not have.
- Past a point you are overfitting to the examples in front of you.
Pick this when: Always first, and again after every other change.
Retrieval
Good at
- Supplying facts, documents and per-user context the model was never trained on.
- Keeping answers current without touching the model.
- Citations, which are often the actual product requirement.
Costs you
- A second system with its own failure modes, latency and bills.
- Retrieval quality becomes the ceiling on answer quality.
- Retrieved text eats context budget that examples and instructions wanted.
- Silent failure: bad retrieval produces a fluent, wrong, confident answer.
Pick this when: The failures are the model not knowing something, rather than not doing something.
Fine-tuning
Good at
- Behaviour that instructions describe badly but examples show well.
- Driving down cost and latency once a task is stable and high volume.
- Getting a small model to do one job as well as a large one does.
Costs you
- Data collection is most of the work and it is not glamorous.
- A training and evaluation loop you now own, per model version.
- Slow to iterate, so it locks in mistakes you have not noticed yet.
- Pins you to a model family at the moment the field moves fastest.
Pick this when: Prompting has plateaued, retrieval is not the gap, and you have or can build a few hundred examples of the behaviour you want.