RAG consulting
RAG consultant: how to judge one before you sign
Retrieval demos well and fails quietly. The gap between a system that answers ten questions in a meeting and one that survives a compliance review is where the money goes, and almost no vendor page describes it.
The search results for this term are almost interchangeable. RAG as a service, agentic RAG, enterprise RAG architecture, dedicated developers available now. They describe a pipeline, they list vector databases, and they stop before the part that decides whether the thing survives contact with real users and a real auditor.
The eight questions
Ask these in the first call. You are testing whether they have shipped one or built one.
- How will we know it is right? The answer has to involve an evaluation set drawn from your questions with known correct answers. "We will test it" is not a method.
- What does a citation point at? A document name is not a citation. A compliance officer cannot verify a claim they cannot follow to a specific version and passage.
- When does it refuse? A system with no refusal path will answer questions it should decline. Refusal is a feature you specify, not a failure you tolerate.
- What happens on an erasure request? Deleting the source document leaves the chunks, the embeddings, the cached answers, the conversation memory and the logs. Ask them to walk the whole chain.
- Which processors touch the data, and where do they sit? An embedding API call is a transfer. So is a reranker. So is anything that logs prompt contents.
- What does re-indexing cost when the corpus changes? Corpora move. A design that assumes a static corpus is a pilot.
- Who owns the vector store and the keys at handover? Same trap as automation credentials.
- Show me a question your system gets wrong. Anyone who has shipped one has a list. Anyone who says none is describing a demo.
Where retrieval builds actually fail
We wrote the long version of this in GDPR-compliant RAG and the three places most builds break it. The short version is worth having in the room during a vendor call.
| Failure | What it looks like in the demo | What it costs later |
|---|---|---|
| Erasure stops at the source document | Nothing. Deletion appears to work | A subject access or erasure request you cannot honour, with derived copies still answering |
| An unmapped transfer | Nothing. The model is hosted in the EU, so the box is ticked | An embedding or reranking call outside the EEA that nobody recorded in the DPIA |
| A citation you cannot follow | Answers look sourced, because a file name appears underneath | A contested answer that cannot be traced, which is the moment trust in the system ends |
| No evaluation set | Ten questions answered correctly in a meeting | No way to tell whether a model or chunking change made it better or worse |
| No refusal path | Confident answers to everything, which reads as strength | A confident wrong answer in front of a customer or a regulator |
What the engagement should cover, in order
- Corpus review before architecture. What the documents are, how they change, whether they contain personal or special category data, and who is allowed to see which parts. Access control at retrieval time is a design input, not a later feature.
- The evaluation set, written before the build. Real questions from the people who will use it, with answers agreed by someone who knows. This is the artefact that makes every later decision measurable.
- A thin end-to-end slice. Ingestion, retrieval, generation, citation and refusal, on a narrow corpus, before anything is scaled.
- The erasure and audit path, tested. Run a deletion through the whole chain and produce evidence it reached the vector store.
- Handover or operation, named explicitly. Who watches quality drift after go-live, and how it gets measured.
On the stack itself, most of the argument is noise. pgvector is the pragmatic default when you already run Postgres and the corpus is in the low millions of chunks, because it removes a system rather than adding one. Qdrant and Milvus earn their operational cost at larger scale or where filtered search performance matters. The vector store is rarely what decides whether the project succeeds. The choice that does matter sits above it: open-weight models you host yourself still trail frontier models on reasoning, and for grounded question answering over a corpus you supply the gap is usually narrow enough not to matter, while for open-ended analysis it is not. Test on your own material before committing to a sovereign stack on principle.
How we work, and what we will refuse
Digiton is an AI consulting firm in Lisbon, building retrieval systems for teams across 8 countries. We will not start a retrieval build without an evaluation set, because without one there is no honest way to report progress and every later conversation becomes a matter of opinion.
We will also say when retrieval is the wrong tool. If the questions have deterministic answers that live in a database, a query is cheaper, faster and correct every time. Retrieval earns its place over documents that resist being turned into rows.
The engineering guidance here comes from deployed systems rather than from a whitepaper, and it is engineering guidance rather than legal advice. Take a finished design to your data protection officer. If you want the buyer's-side version of this page, it is in what to look for in a RAG vendor, and the sector-specific version for legal teams is in enterprise RAG for legal teams.
Frequently asked questions
What does a RAG consultant do?
Designs and builds retrieval over your own documents, and the honest version covers four deliverables: an evaluation set built from your questions with known correct answers, citations that resolve to a document version and passage, a specified refusal path, and an erasure procedure that reaches chunks, embeddings, cached answers and logs. A proposal missing any of those is describing a demo.
How do I know whether a RAG system is any good?
By measuring it against an evaluation set drawn from real questions with answers agreed by someone who knows the material. Without that set there is no way to tell whether a model change, a chunking change or a reranker made the system better or worse, and every review becomes a matter of opinion.
What makes a RAG system GDPR-compliant?
Three properties, none about model hosting. Erasure has to reach every derived copy, meaning chunks, embeddings, cached answers, conversation memory and logs. Every transfer outside the EEA has to be identified and covered, and an embedding API call is a transfer. And every answer has to trace to a specific document, version and passage, because a compliance officer cannot verify what they cannot follow.
Which vector database should we use?
pgvector is the pragmatic default when you already run Postgres and the corpus is in the low millions of chunks, because it removes a system rather than adding one. Qdrant and Milvus earn their operational cost at larger scale or where filtered search performance matters. The vector store is rarely what decides whether the project succeeds.
Should we self-host the model?
Test before deciding on principle. Open-weight models you host yourself still trail frontier models on reasoning quality. For grounded question answering over a corpus you supply, the gap is usually narrow enough not to matter. For open-ended analysis it is not. Run both on your own material and read the results.
How long does a RAG build take?
The order matters more than the duration. Corpus review, then the evaluation set written before any building, then a thin end-to-end slice covering ingestion, retrieval, generation, citation and refusal on a narrow corpus, then the tested erasure path, then scale. Teams that skip straight to scale rebuild.
When is RAG the wrong tool?
When the questions have deterministic answers that live in a database. A query is cheaper, faster and correct every time. Retrieval earns its place over documents that resist being turned into rows, and a consultant who never says this is selling rather than advising.
Related
Want the number before the call?
The cost calculator gives you a build band, a monthly running cost and a payback range from our published 2026 price list. No email needed to see it.
Open the AI automation cost calculatorTell us the operation, get a scoped answer
One line about the work you would hand over. We reply within one working day with the band it falls in and whether it is worth building at your volume.