Fine-tuning versus prompting: what a small business actually needs
Vendors talk about training a model on your data. Almost every small business need is met by good instructions and retrieval instead.
The short answer
There are three ways to make a general AI model useful for your business, and they differ enormously in cost and commitment. Prompting means giving the model clear instructions and examples every time it is used. Retrieval means giving it your own documents at the moment of the question. Fine-tuning means changing the model itself with thousands of your examples. Almost every small business need is met by the first two: the model already knows how to write, summarise and reason; what it lacks is your facts and your rules, which prompting and retrieval supply. Fine-tuning is for a narrow, high-volume task with a specific output shape, and only when prompting has been tried and measured first.
The three approaches
| Prompting | Retrieval | Fine-tuning | |
|---|---|---|---|
| What it changes | The instructions the model receives | The information the model receives | The model’s internal weights |
| Good for | Tone, rules, format, task definition | Your facts, documents, current data | Narrow tasks with a fixed output shape at volume |
| Cost to set up | Hours | Days | Weeks, plus data preparation |
| Cost to change | Edit text | Update a document | Retrain |
| Reflects a document update | Immediately, if included | Immediately | Only after retraining |
| Teaches facts reliably | For facts in the prompt | Yes | No |
| Lock-in | Low; prompts move between models | Low; the index is yours | High; tied to one model version |
| Typical small business use | Nearly everything | Anything answering from your data | Rare |
Deciding, in order
- Write the instructions well. Role, rules, format, examples of good output. Most quality problems are solved here.
- Add retrieval if the task depends on your documents or data. The model answers from what it is shown.
- Measure against a set of real cases with known good answers. If it meets the bar, stop.
- If it falls short in a specific, repeatable way, on a narrow task, at high volume, with thousands of vetted examples available, evaluate fine-tuning against the same measurement.
- Even then, keep retrieval for facts. Fine-tune the behaviour; retrieve the knowledge.
What we do in practice
For clients we start with a well-written instruction set and, wherever the task touches company knowledge, retrieval from documents the client controls. We measure against real cases before anything goes live. We have rarely needed fine-tuning for a small or mid-sized business, and when the case arises it is a narrow classification or extraction task at volume, evaluated against the prompted baseline it has to beat.
What this means for you
Start with good instructions. Add your documents through retrieval. Measure. That handles almost everything a small business asks of AI, cheaply, changeably and without lock-in. Fine-tuning is a tool for a specific, narrow, high-volume job, chosen on evidence after prompting has been measured, never as the first step and never as the way to teach a model your facts.
Frequently asked questions
A vendor says our assistant needs to be trained on our data. Is that true?
Usually not, and the phrase often means retrieval anyway. An assistant that answers from your documents needs those documents indexed and retrieved at question time, not baked into the model. Ask the vendor precisely what they mean and what changes when a document is updated; if the answer is retraining, the design is wrong for a business whose documents change.
When would fine-tuning actually be worth it?
A narrow task done at high volume where the output must follow a very specific style or structure, you have thousands of vetted examples, prompting has been tried and falls short, and shaving cost per call matters at your volume. Classification of support tickets into your categories is a typical case. A general assistant is not.
Does fine-tuning make the model know our business?
It makes the model imitate the style and patterns of your examples. It is unreliable for teaching facts, and facts change. Knowledge belongs in retrieval, where a document update is immediately reflected. Fine-tuning for facts produces a model that confidently states last quarter's prices.