A bigger model or a better prompt
Quality is short of what you need. Do you move up a model tier, or spend the time on the prompt you already have?
The short answer
Read the failures first. If they are ambiguity, fix the prompt; if they are genuinely hard instances, buy the bigger model — and consider routing, so only the hard ones pay for it.
Move up a tier
Good at
- Failures of reasoning, multi-step arithmetic, and long instructions the small model drops.
- Being a single change you can measure in an afternoon.
- Raising the ceiling when careful prompting has stopped helping.
- Buying reliability on rare hard inputs, which is often where the complaints come from.
Costs you
- Several times the price per token, on every request including the easy ones.
- Higher latency, typically noticeably slower to first token and per token.
- It can hide the real bug — an ambiguous instruction that the larger model happens to guess right.
- Gains often shrink once the prompt is fixed, so you paid for both.
Pick this when: The failures are on genuinely hard instances, and a careful prompt on the smaller model has already been tried and measured.
Fix the prompt
Good at
- Being free at inference time, forever.
- Fixing the most common real cause: the model did what you asked, and you asked wrong.
- Surfacing the actual specification, which the team probably has not agreed on either.
- Transferring — a clearer prompt usually helps on the bigger model too.
Costs you
- Your time, and it is easy to spend a day for nothing without an eval set.
- There is a ceiling, and past it more prompt engineering makes things worse.
- Long prompts have their own cost in tokens and latency.
- Improvements are hard to tell from noise on small eval sets.
Pick this when: Always first. Read twenty failures before changing anything; most of them will not be capability failures.