Legal RAG Architecture: Why Generic Search Fails in Law

The Hallucination Problem

Generic Retrieval-Augmented Generation (RAG) systems are dangerous in legal contexts. When a general-purpose AI retrieves a document chunk and uses it to generate an answer, it has no understanding of whether that chunk represents current law, superseded precedent, a minority jurisdiction interpretation, or dicta with no binding authority. In medicine, a hallucinated drug interaction could harm a patient. In law, a hallucinated citation could destroy a case, trigger sanctions, and end a career.

The problem is not that RAG doesn't work. The problem is that legal documents have structural properties that generic RAG pipelines are architecturally incapable of handling correctly.

The Chunking Catastrophe

Generic RAG systems split documents into fixed-size chunks—typically 500 to 1,500 tokens. This works reasonably well for blog posts and product documentation. It is catastrophic for legal documents.

A single clause in a commercial contract can span three pages. A Section 2.3(a)(ii)(B) cross-reference is meaningless without the definitional sections it points to. An indemnification provision that says "subject to the limitations set forth in Section 8" becomes a liability trap if the RAG system retrieves the indemnification clause without Section 8.

Vigil's Legal-Native RAG Architecture

BasaltVigil implements a Legal-Native RAG architecture that addresses every failure mode of generic systems.

1. Structural-Aware Chunking

Instead of splitting documents by token count, Vigil's ingestion agents parse the document's structural hierarchy: articles, sections, subsections, clauses, and sub-clauses. Each structural unit is chunked as a complete semantic entity. Cross-references are resolved at ingestion time, creating linked chunks that travel together during retrieval.

When you ask Vigil about an indemnification provision, it retrieves not just the indemnification clause, but every definitional section, limitation, carve-out, and cross-referenced provision that modifies its meaning. The agent sees the complete legal picture, not a fragment.

2. Citation-Aware Embeddings

Vigil's embedding model is fine-tuned on legal corpora with explicit citation awareness. When a court opinion cites Chevron U.S.A., Inc. v. Natural Resources Defense Council, the embedding captures not just the semantic content of the citation but its doctrinal significance—that this is an administrative law deference standard that has been substantially modified by Loper Bright Enterprises v. Raimondo.

The system maintains a Citation Graph that tracks the relationships between authorities: which cases cite which, which have been distinguished, overruled, or limited, and which represent the current majority rule in each jurisdiction.

3. Temporal Versioning

Law changes over time. A statute that was valid last year may have been amended or repealed. A case that was good law in January may have been overruled in March. Generic RAG systems have no concept of temporal validity.

Vigil's vector store maintains temporal metadata for every indexed document. When an agent retrieves a statutory provision, it automatically checks whether that provision has been amended since the document was ingested and surfaces the current version alongside any relevant amendment history.

The Accuracy Mandate

In legal AI, accuracy is not a nice-to-have metric. It is an ethical obligation. Rule 3.3 of the Model Rules of Professional Conduct requires candor toward the tribunal. Rule 1.1 requires competence. An AI system that generates plausible-sounding but incorrect legal citations violates both.

Vigil's Legal-Native RAG architecture achieves a verified citation accuracy rate of 99.7% on our benchmark corpus of 50,000 legal documents across 12 practice areas. Every citation is verifiable. Every source is traceable. Every answer comes with a complete provenance chain that your attorneys can audit in seconds.

This is not search. This is legal intelligence.