Fine-Tuning and Customization

There are three ways to make a general model behave the way you need, in ascending order of effort: prompting (instructions and examples in the context), retrieval (inject your data at query time, from Level 3), and fine-tuning (further training the model's weights on your own examples). The most valuable instinct at this level is knowing which to reach for — and it is usually not fine-tuning.

The rule of thumb: prompt first, add retrieval, fine-tune last. Prompting and RAG are fast, cheap and trivially reversible — you change a string and try again. Fine-tuning costs money and data engineering, takes time, and bakes behaviour into weights so every update means retraining. Reaching for the heaviest tool first is the classic expensive mistake, and it's usually made because fine-tuning sounds more serious than writing a better prompt.

Be precise about what fine-tuning is actually good for. It excels at form: a consistent house style, a rigid output format, a domain vocabulary and register that prompting keeps drifting away from. It excels at specialisation — taking a small cheap model and lifting it to match a much larger one on one narrow task, which is often the strongest cost argument for doing it at all. And it can encode behaviour that would otherwise need a very long prompt, saving those tokens on every single call.

It is the wrong tool for knowledge. Facts change; weights don't. Fine-tuning a model on your product documentation gives you a model that has absorbed a snapshot in a lossy, unciteable way — it will still be confidently wrong when the documentation changes, and it can't tell you where an answer came from. Retrieval is the right answer for facts, precisely because it's updateable and citable. If you find yourself planning to fine-tune so the model "knows about our product", that's the signal to build RAG instead.

When you do fine-tune, example quality dominates volume. A few hundred clean, consistent, genuinely representative examples routinely beat several thousand noisy ones, because the model will faithfully learn whatever inconsistency is in your data — including mistakes. Curate deliberately: cover the range of cases you actually see, keep the format uniform, and have a human check a sample. Hold out a test set from the start, and keep it out of training.

Know the failure modes before you start. Overfitting shows up as a model that's excellent on data resembling your examples and worse than the base model on anything else. Catastrophic forgetting is the subtler one: tuning hard on a narrow task can degrade general capability you were relying on. Both are invisible unless your evals cover more than the target task, which is the argument for having built them in Level 3.

Consider the lighter options in between. Parameter-efficient methods like LoRA train a small number of additional weights rather than the whole model, which is dramatically cheaper, faster to iterate on, and easy to swap or roll back. For most teams that's the practical form of fine-tuning, and it takes the decision from a major project to an experiment you can run in a week.

Count the ongoing cost honestly, not just the training run. A fine-tuned model is a thing you now own: it needs re-tuning when the base model is deprecated, its own evals, its own deployment path, and someone who remembers how it was built. Providers retire model versions, and a fine-tune anchored to a retired base is a migration you didn't plan.

The discipline is simple to state and hard to follow. Exhaust prompting. Add retrieval. Prove with evals that you've hit the ceiling of both. Only then fine-tune — for style, format and specialisation, never for facts.