When fine-tuning beats prompting (and when it doesn't)

Fine-tuning gets reached for earlier than it should be. Most quality problems that look like "the model needs to be trained on our data" are actually retrieval problems or instruction problems, and both are cheaper and faster to fix than a training run.

Try prompting and RAG first

If the model doesn't know something, that's a retrieval problem — RAG solves it more cheaply than fine-tuning and updates instantly when the underlying information changes, where a fine-tuned model has to be retrained. If the model knows the information but responds in the wrong format, tone, or level of detail, that's usually a prompting or few-shot-example problem, not a training problem. Fine-tuning is the wrong tool for both of these, even though both can look like "the model just doesn't get it" from the outside.

Where fine-tuning actually earns its cost

Fine-tuning is the right call when the task requires internalizing a pattern that's too complex or too voluminous to express in a prompt — a specific structured output format applied consistently across thousands of variations, a domain-specific vocabulary and style that few-shot examples can't reliably convey, or a latency/cost requirement where a smaller fine-tuned model needs to match a larger general model's quality on a narrow task. It also pays off when the same narrow task runs at high enough volume that the training cost amortizes against inference savings from using a smaller model.

A simple ordering

In practice: reach for a better prompt first, add RAG if the model is missing information rather than misapplying a format, and reach for fine-tuning only once both of those have been tried and the gap is specifically about consistency, style, or cost at scale — not raw knowledge.