Fine-tuning or few-shot prompting
You have a few hundred examples of the behaviour you want. Put them in the prompt, or train on them?
The short answer
Few-shot until it plateaus. Then tune, using the plateaued prompt's outputs and your corrections as the first version of the training set.
Few-shot prompting
Good at
- Working immediately — add examples, rerun the eval, see the result in minutes.
- Being editable: one bad example can be deleted, not retrained away.
- Surviving a model change, usually with minor adjustment rather than a rerun.
- Prompt caching, which makes a long fixed example block far cheaper after the first call.
- Requiring few examples — five to twenty well-chosen ones do most of the work.
Costs you
- Tokens on every request, and with caching still not free.
- Example order and selection matter more than people expect, and tuning them is fiddly.
- Only a handful of examples fit usefully, so rare cases go unrepresented.
- Long example blocks push out the room retrieved context wanted.
- Consistency across a long output degrades in a way tuned models resist.
Pick this when: Always first. If twenty examples do not move the eval, more examples are unlikely to be the problem.
Fine-tuning
Good at
- Using all your examples — thousands, not twenty.
- Consistency, especially on format, which is where tuned models most clearly win.
- Cutting per-request cost once the examples leave the prompt.
- Lower latency, since prefill shrinks with the prompt.
- Letting a much smaller model do the job, which compounds the cost saving.
Costs you
- Data quality is the whole game, and cleaning a dataset is slow unglamorous work.
- A training run per change, so iteration is hours rather than minutes.
- Errors in the data become behaviour you cannot prompt your way out of.
- A model artefact to version, evaluate and serve.
- A new base model means doing it again.
Pick this when: Few-shot has plateaued, volume is high enough that prompt tokens are a real bill, or you need consistency that instructions are not delivering.