Insights · AI
RAG and governance: reliable AI answers come from your sources, not the model’s guesses. Retrieval, rules and human control cut hallucinations. See how it works.
A language model doesn't consult an archive when it answers: it generates the most probable sequence of words given the context. When a question touches information that isn't in its training data, or that changed after training, the model doesn't stop and doesn't say 'I don't know'. It keeps producing plausible text, with the same confident tone it would use for a correct answer. This is the mechanism behind so-called hallucinations: they aren't an occasional bug to patch, but the natural behaviour of a tool that optimises for fluency, not for truth.
The worst risk isn't the obviously wrong answer, which you spot immediately. It's the believable one: a regulatory reference that doesn't exist, a contract clause cited confidently but invented, a date or a figure inserted to complete the sentence. In a business context this turns a useful tool into a source of operational and reputational risk. Asking the model to 'be accurate' doesn't fix it, because the model has no way to tell what it remembers from what it builds on the spot.
The breakthrough isn't choosing a bigger model, but changing where the answer comes from. Instead of letting the model draw on its statistical memory, you hand it the relevant material at the moment of the question and ask it to answer only on that basis. The model stops being the archive and becomes the reader who synthesises. This is the idea at the root of RAG, retrieval-augmented generation: retrieve first, then generate.
When a question arrives, the system doesn't hand it straight to the model. First it searches your documents, pages, contracts, procedures, product sheets, for the passages most relevant to that specific request. This retrieval phase is the heart of the mechanism: if it pulls the right material, the answer will have a foundation; if it pulls imprecise or absent material, no model can compensate. The quality of retrieval matters more than the power of the model.
Only now does the model enter, and it receives a precise instruction: answer using only the passages provided; if the information isn't there, say so. The difference from open-ended use is stark. The model no longer has to guess: it has to read, select and reformulate what's in front of it. Its linguistic talent is put at the service of your documentation, not its imagination.
A serious answer points to where it comes from. Every important claim should be traceable to the document passage it was drawn from, so a person can verify it in seconds. Citation isn't a cosmetic detail: it's what makes the answer auditable. Without traceability, even a correct answer remains an act of faith; with traceability, it becomes a verifiable statement.
The system answers only from a defined, up-to-date set of documents, not from the whole web nor from the model's internal memory. Narrowing the perimeter of sources is the first way to keep answers from drifting toward unverified information.
Abstention has to be instructed and rewarded: when the sources don't contain the answer, the system must say so instead of filling the gap. A 'this isn't in the available documents' is worth more than an invented answer.
When a policy changes, you update the document and the answer changes with it, without touching the model. This keeps the system aligned with the company's current reality and reduces answers based on outdated information.
Every answer carries its source references, so the reader can check independently. Verifiability turns blind trust into informed trust and makes it possible to correct the system when it errs.
Giving the system access to company documents raises a question of control: not everything that's retrievable should be retrievable by anyone. The permissions of whoever asks the question must also apply to retrieval. A team member shouldn't be able to obtain, through a synthesised answer, information they couldn't read by opening the original document. Access governance isn't a layer above the system: it's part of the system.
When sources contain personal data, GDPR principles apply as they would to any other processing: legal basis, minimisation, defined purpose, limited retention. That means deciding consciously what goes into the retrievable archive and what doesn't, and handling sensitive categories with particular care. The question isn't only 'does the system answer well', but 'is it using data it has the right to use, for the purpose it was collected for'.
A governed system leaves a trail: which sources were consulted, what answer was given, when. This serves to verify, to correct and to demonstrate compliance, but it in turn must be disciplined, because logs too can contain personal data and need a retention policy. Mature governance doesn't just make the system work: it defines who is responsible for what, and for how long information stays available.
Before talking about models, you decide which documents the system may draw on, who owns them and how often they're updated. A clear perimeter is the premise of every reliable answer.
You check who can see what and where personal data lives, aligning the system's access with people's real access and applying minimisation principles from the design stage.
You configure the system to answer only from retrieved material and to declare the absence of information when the sources fall short. Abstention is treated as a correct answer, not a failure.
For the highest-impact cases, the system's answer is a draft a person reviews before it becomes final. Human control doesn't slow everything down: it concentrates where an error would cost the most.
You observe real answers, verify citations, find the weak points of retrieval and act on the sources. The system improves by updating documents, not by chasing ever-bigger models.
An ordered path, not a switch to flip on
Does RAG eliminate hallucinations entirely?
No, it reduces them substantially but doesn't zero them out. Grounding the model in your sources makes invention far harder and makes every answer verifiable, but a margin of error remains: imprecise retrieval or a forced synthesis can still produce inaccuracies. That's why human control over critical cases and the ability to verify citations matter.
Do we need to retrain the model on our data?
Almost never, and it's rarely the first thing to do. In most cases it's more effective and safer to supply the documents at the moment of the question: you update a source and the answer changes, with no work on the model and without baking your data in permanently. It's also a more privacy-respecting approach, because the data stays in the sources you control.
What happens if a document is wrong or outdated?
The system will answer according to that document, because it trusts the sources you give it. That's the central point: answer quality depends on the quality and freshness of the archive. This is why source maintenance, retiring what's obsolete and updating what changes, is an integral part of the system and not an optional extra.
Can we control who sees which information?
Yes, and it's a requirement, not an option. Retrieval must respect the permissions of whoever asks: no one should obtain through an answer what they couldn't read directly. Together with minimisation of personal data and a retention policy for logs, this is what makes the system compatible with GDPR and with your internal rules.