RAG vs fine-tuning. Give the model the right documents, or change its habits.
RAG and fine-tuning are two ways of adapting a language model to the work of a business. The first has it read the right documents at the moment of the question; the second retrains it on examples. For almost every small business, the real question is whether the second is needed at all.
RAG changes what the model has in front of it when it answers; fine-tuning changes how it answers. With the first, your company data stays in your documents; with the second, it goes into the model.
The term comes from a paper by Patrick Lewis and others, presented at NeurIPS in 2020: models that combine the memory locked in their parameters with an external memory to consult, so they can cite the source and update what they know.
You almost always start with instructions and RAG. OpenAI writes that prompt work may be enough on its own; fine-tuning is for when format or style must stay exactly the same and good examples are plentiful.
Written by: Mattia Esposito, AI and marketing consultant, founder of Itria AI. From Puglia, based in Turin since 2023, he works remotely in Italy and abroad. LinkedIn profile.
The RAG and fine-tuning entries are defined in the glossary. Here the point is the choice: what changes between the two, when you need one or the other, and what it means for your data.
The difference, row by row
The two techniques solve different problems, which is why they get mixed up: both promise a model that knows your business. The table puts side by side the eight questions an owner actually asks.
| Question | RAG | Fine-tuning |
|---|---|---|
| What changes | What the model reads before it answers. | The model itself, retrained on examples. |
| Where your data sits | In the documents, which the system consults. | In the model's parameters, after training. |
| The price list changes | You replace the document. | You retrain, or the model falls behind. |
| Citing the source | Yes: the answer can say which document it comes from. | No: the model doesn't know which example it learnt from. |
| What you need to start | Tidy, up-to-date documents. | Hundreds of good question-and-answer examples. |
| When to use it | When the model doesn't know your data. | When the model won't answer in the format or style you want, despite the instructions. |
| Typical failure | Outdated sources, read with confidence. | A model that repeats the mistakes in the examples. |
| Personal data | Removed by removing the document. | Goes into the model, and you have to assess whether it can be extracted. |
What is RAG
RAG stands for retrieval-augmented generation: before writing, the system searches the authorised documents and uses them. The 2020 paper that gave the technique its name started from a limit of language models: providing provenance for their answers and updating what they know “remain open research problems”.
In a business the documents are the price list, the spec sheets, the procedures, the terms of sale. The system finds the right passage and answers from it, and the person rereading can check the source. It's the first of the four capabilities of an AI agent: retrieving from authorised sources.
What is fine-tuning
Fine-tuning takes a model that has already been trained and trains it further on your examples, until it answers like those examples. It's used to lock in a behaviour: an unchanging format, a tone, a classification repeated thousands of times.
OpenAI's documentation on model optimisation puts fine-tuning at the end of a cycle that starts with evaluations and instructions, and on instruction work it says: “The prompt engineering process may be all you need”. Fine-tuning comes later, when instructions aren't enough and the examples to train on exist.
Which one a small business needs
If the problem is that the model doesn't know your data, you need RAG; if it won't answer in the form you want despite the instructions, fine-tuning may help. In most small businesses the problem is the first, and it's solved with clear instructions and tidy documents.
Fine-tuning makes sense for narrow, repeated tasks, with hundreds of examples already corrected by hand: sorting incoming enquiries into your own categories, writing a product description always in the same pattern. Even there, it's worth trying instructions and examples in the prompt first, because it costs less and can be changed in a minute.
An example: customer questions on prices and formats
A distributor asks the price of a format and the minimum order: the textbook case for RAG. The system searches the current price list and the product's spec sheet, drafts the reply with those figures and says which document they came from; a person rereads it before it goes out. When the price list changes, you replace the file and the next day's reply uses the new prices.
Doing the same job with fine-tuning would mean retraining the model for every price list, and the model couldn't say where a price comes from. Fine-tuning would earn its place on another part of the same job: sorting incoming enquiries into your categories, if there are thousands already sorted by hand.
Three questions before choosing
Three questions are enough to see where to start. Does the model get it wrong because it doesn't know a fact, or because it doesn't put it the right way? Does that fact change more than once a year? Are there already hundreds of examples corrected by hand?
If the model doesn't know the fact and the fact changes, you start with RAG. Fine-tuning comes in only if instructions can't produce the form you want and the corrected examples already exist. If none of the three answers leads there, the work to do is better instructions and tidier documents.
It's a choice more and more businesses face. According to Istat, 16.4% of Italian companies with at least 10 employees used at least one AI technology in 2025, up from 8.2% in 2024.
Personal data: what changes between the two
With RAG, personal data stays in the documents, and removing a document removes it from the answers; with fine-tuning it goes into the model. The European Data Protection Board, in Opinion 28/2024 of 18 December 2024, says that whether a model trained on personal data is anonymous has to be assessed case by case by the authorities.
For it to be anonymous, the opinion adds, it must be very unlikely that the people whose data was used to build it can be identified, or that their data can be extracted through queries. For a small business the practical rule is simple: personal data stays out of training examples, and where it's needed you work with RAG on documents that can be deleted.
What the risk guidance says
NIST, the US government's standards institute, treats the two techniques as choices to document. In its July 2024 profile on generative AI, NIST AI 600-1, it suggests measuring a model's baseline performance before choosing it for fine-tuning or RAG, and documenting how much human knowledge enters the system through either.
In practice: first you measure how the model answers with nothing added, then you add the minimum needed. Anyone proposing fine-tuning as the first step should be able to say what instructions and documents couldn't do.
Related terms
LLM
The language model that RAG and fine-tuning each adapt in their own way.
AI hallucinations
A false statement made with confidence. RAG reduces it when the sources are right, and makes it worse when they're outdated.
Knowledge base
The set of documents a RAG system draws its answers from.
Embedding
The numerical representation of text that lets RAG find the right passage.
Questions and answers
What is RAG?
RAG stands for retrieval-augmented generation: before answering, the system searches the authorised documents and answers from them, and it can say which document the answer comes from.
The term comes from a paper by Patrick Lewis and others presented at NeurIPS in 2020.
What is fine-tuning?
It's further training of an already trained model on your own examples, so that it answers the way those examples do.
It's used to lock in a format, a tone or a repeated classification, and it needs hundreds of good examples.
What's the difference between RAG and fine-tuning?
RAG changes what the model reads before it answers; fine-tuning changes the model itself.
With RAG a new price list is updated by replacing the document; with fine-tuning you have to retrain.
Does RAG eliminate hallucinations?
No. It reduces them when the sources are right and up to date, because the model answers from a document rather than from memory.
With outdated sources the wrong answers come out just as confidently, which is why a person rereads whatever goes out to a customer.
Does a small business need fine-tuning?
Almost never as a first step. You start with clear instructions and RAG on the company's documents.
Fine-tuning makes sense for narrow, repeated tasks, when there are hundreds of examples already corrected and the instructions aren't enough.
Notes on sources
- Patrick Lewis and others, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, NeurIPS 2020, first version of 22 May 2020; the quotation comes from the abstract.
- OpenAI, Model optimization, developer documentation read on 8 November 2026, the source of the quotation on instruction work.
- European Data Protection Board, press release of 18 December 2024 on Opinion 28/2024 on AI models.
- NIST, Generative Artificial Intelligence Profile, NIST AI 600-1, July 2024, actions MS-2.3-001 and MS-2.5-002.
- Istat, Imprese e ICT (ICT in enterprises), 2025, covering companies with at least 10 employees.
- This page is a technical explanation, not legal advice on processing personal data.
First you see what the model doesn't know. Then you decide what to give it.
The first step with Itria is a thirty-minute video call: we look at which task you want to hand to an AI system, which documents already exist, and where a person needs to approve before anything is sent. Drop us a line about what's slowing you down.