RAG for legal teams
Enterprise RAG for legal teams
Legal is the one department where a plausible answer with no source is worse than no answer at all, which makes it the best possible test of a retrieval system.
In-house legal teams sit on contracts, policies, precedent, advice notes and correspondence that nobody has read as a whole. Retrieval augmented generation is the right tool for that estate, and it fails in specific, predictable ways when it is built the way general purpose chatbots are built.
Chunk at the clause, not at the character
Fixed length chunking splits a limitation of liability clause across two chunks and returns half of it. Legal documents have structure worth respecting: clause and sub-clause boundaries, defined terms, schedules, and the amendments that change all of the above. Chunk on that structure, carry the document metadata (party, date, governing law, status, version) on every chunk, and store the amendment chain so the system can answer with the operative version rather than the first one it found.
Access control belongs in the index
Privilege and matter confidentiality are not a filter you apply to results. If a user should not see a document, its chunks should never enter their retrieval set. That means access rules enforced at the database level with row-level security, evaluated per query, and traces recorded per user. A system that retrieves everything and filters afterward will eventually leak through a summary.
Citation that can be checked in one click
Every assertion needs a link to the clause it came from, and the interface should make checking it faster than not checking it. This is the single change that moves a legal team from suspicion to use. It also changes the failure mode: an unsupported claim becomes visibly unsupported rather than quietly wrong.
The evaluation set is the deliverable
Build a set of questions with known correct answers and known source clauses, written by the lawyers who will use the system. Rerun it on every model change, every chunking change, and every index rebuild. Without it, quality is an opinion that varies by whoever last tried it. Somewhere between 100 and 300 questions covering the awkward cases is usually enough to catch regressions.
What it does not do
It does not give advice, and it should not be sold as doing so. It finds the operative clause, surfaces the inconsistency across a portfolio, and drafts the first version of an answer a lawyer then owns. That is a large amount of value and an honest description of it. Digiton builds retrieval systems this way and operates them afterward. An AI audit is the usual starting point.
Frequently asked questions
What is enterprise RAG for a legal team?
Retrieval augmented generation over the organisation own legal documents: contracts, policies, precedent and advice. The system finds the relevant clauses and drafts an answer with citations back to the source, so a lawyer can verify it in one click rather than trusting an unsourced summary.
How do you keep privileged documents out of results?
Enforce access at the index, not on the output. Rules run at the database level with row-level security so restricted chunks never enter a user retrieval set, evaluated per query and logged per user. Filtering after retrieval eventually leaks, usually through a summary that nobody traces back.
Can a RAG system give legal advice?
No, and any vendor selling it that way is creating a liability rather than solving one. It locates the operative clause, surfaces inconsistencies across a portfolio, and drafts a first version. A qualified person owns the advice. The value sits in the search and the drafting time, which is substantial on its own.
Related
Ready to put AI to work?
Book a discovery audit and we will map the highest-ROI AI agents and automations for your business.
Book a discovery audit →