AI Due Diligence Data Room Review: From Index to Request List

Collapsing the work into "ask the data room a question" is the root error: three distinct jobs get routed through one retrieval step that silently decides which job it is doing.

Index reconciliation is set arithmetic. What did we ask for, what arrived, what arrived unrequested, what arrived unsigned. The output is a delta, not an answer, and it needs no reading comprehension at all.

Thesis-driven extraction is targeted reading. Every assignment and change-of-control provision, every exclusivity and most-favoured-nation term, every earn-out trigger, scored by materiality. The output is a ranked list with citations.

Contradiction hunting is comparison. The disclosure schedule, the board minutes and the customer contract each say something internally consistent and mutually incompatible. No single document contains the finding.

Three inputs, three outputs, three failure modes. A tool that answers all three from one similarity search returns a fluent paragraph for each and is wrong in a different way each time. The stakes are not abstract: studies of private-target acquisitions running into the thousands of deals report the buyer's proposed post-close working capital calculation accepted without dispute in roughly seven of ten cases. Diligence findings land in price mechanics, so a missed contract term has a dollar consequence and not just a legal one.

The first deliverable is not a summary. It is a delta report, shipped before anyone reads anything for substance.

The pipeline classifies every file into a document type and extracts the named parties, execution status, effective and termination dates, and a version lineage linking amendments and restatements back to their parent agreement. Then it maps each artifact to a request line item and emits the gaps.

The unparseable status is the one teams leave out and then regret. A failed parse that silently becomes a "requested, not provided" line turns an infrastructure problem into a false finding, and false findings burn credibility with the other side faster than missed ones.

Extract first and fix entities later, and you will produce a change-of-control finding attached to the wrong legal entity. That is worse than no finding, because it looks like work product.

The room contains a group: a topco, holding companies, operating subsidiaries across jurisdictions, trading names that appear in no registry, an entity renamed after an acquisition, and signature blocks using a fourth variant. Resolve all of it into one canonical graph before the reading starts.

Four inputs feed the graph. An alias table seeded from the corporate records in the room. Signature-block parsing, the highest-signal source because it names the entity that actually bound itself. Counterparty names from contract preambles, which frequently differ from the executing entity in the same document. And the org chart, reconciled against the registration numbers in the constitutional documents.

The rule we hold to: never auto-merge on name string similarity alone. Auto-merge requires two independent corroborating signals, for example a matching registration number plus a matching registered address, or a signature block naming both variants in one instrument. Everything else goes to an adjudication queue where a human confirms or splits, and every adjudication updates the alias table so the same ambiguity is not asked twice. This is a graph problem and behaves like one: we covered the pattern in building a knowledge graph over enterprise data, and the entity-resolution mechanics in that knowledge graph platform case study transfer directly to a group structure.

The reference dataset for contract review, CUAD (arXiv:2103.06268), is 510 commercial contracts from public filings, hand-annotated with more than 13,000 labels across 41 clause categories. It is the right benchmark and the wrong operating model.

Building it cost an estimated two million dollars of senior legal time: 9,283 pages, each reviewed at least four times, at five to ten minutes per page. Every annotator went through 70 to 100 hours of training first, and every label was verified by three more. That is the calibration cost the pipeline exists to amortize, and the reason an unreviewed extraction is not a finding.

It is also the reason not to run all 41 categories on every document. A generic sweep returns every match at equal weight and buries the six provisions that decide the deal. Derive the extraction list from the thesis instead. Customer expansion leads with customer concentration, assignment and change of control, exclusivity, MFN and termination for convenience. A carve-out leads with shared-services dependencies, IP assignment chains and data-processing terms. Where there is an earn-out, every trigger definition and accounting-policy reference is material by construction.

Then score by materiality rather than returning matches flat. The score combines contract value against the disclosure schedule's own threshold, the counterparty's share of revenue, remedy severity, and whether a cure period exists. Above the line is read in full. Below it is sampled.

On expected quality: vendor-published extraction benchmarks cluster around 0.85 F1 on M&A provisions, self-reported on the vendor's own test set. The missing fraction is the interesting part. A 2025 benchmark of four proprietary and fifteen open-weight models on clause-level legal risk identification (arXiv:2508.03080) found open models frequently answering "no related clause" when a relevant clause was present. That is a silent false negative: nothing appears in the report, so nothing prompts a second look. It is the most dangerous failure mode in diligence, and it is invisible unless you sample the non-hits.

Picture a customer relationship documented in four places. The master agreement requires consent for assignment. An amendment two years later carves out transfers to affiliates. A side letter to the same customer reinstates consent specifically on a change of control. The disclosure schedule lists that customer under "consent not required". Every document is internally consistent. The set is not, and the answer determines whether the buyer inherits the contract.

You cannot find this with one long-context pass. NoLiMa tested twelve models advertising 128K-plus windows and found that once the answer stopped sharing vocabulary with the question, ten of the twelve had dropped below half their own short-context baseline by 32K tokens (arXiv:2502.05167); GPT-4o fell from 99.3% to 69.7%. A data room is orders of magnitude past 32K, and diligence questions are exactly the lexically-mismatched kind. We wrote up the degradation pattern in long-context degradation and context rot.

Build a claim graph instead. Extract atomic claims with their source anchor and subject entity, index by subject and topic, and run pairwise conflict checks only across claims sharing a subject. When a candidate surfaces, re-retrieve the surrounding clause on both sides before confirming, because most first-pass conflicts dissolve once the governing definitions are in view. Contradiction detection is its own measured task with its own error profile, not a byproduct of good retrieval: work on LLMs as context validators found it hard for state-of-the-art models, with performance varying sharply by conflict type (arXiv:2504.00180), and the graph-shaped benchmark for inter-context conflicts (arXiv:2507.21544) is what to point your evaluation at. The claim graph is not an implementation quirk. It is the shape the measured task has.

Two calibration points sit behind that table. The first head-to-head benchmark of legal AI tools against a lawyer control group found the tools ahead on extraction and document question answering, level on chronology generation, and behind on open-ended research against public filings, where the human baseline was about 70 percent and the one tool scored in that category came in around 55 percent. Extraction is where the machine wins, open-ended research is where it does not, and a pipeline that blurs the two gets trusted in the wrong place. Second, the clause-level benchmark above also found reasoning mode improving output usefulness while reducing correctness, and quantization buying speed at a measurable accuracy cost.

A workflow tool, or an RPA job, assumes three things: a stable input schema, a fixed step order, and an output another system consumes. Invoice processing has all three. A data room has none. The input is a pile of mixed formats where the same commercial term lives in a signed PDF, a scanned side letter and a spreadsheet tab. The step order is discovered, not designed: whether you read the board minutes depends on what the shareholders' agreement said about reserved matters. And the output is not a database row. It is a work product a senior person signs and someone may litigate.

That combination is what makes this agent work rather than automation. The system plans a reading order, maintains state across thousands of documents, forms a hypothesis, re-retrieves against its own hypothesis, and escalates when confidence drops. It is the architecture we describe across the AI agents pillar, applied to a corpus that changes under you every time the seller uploads. Our diligence analyst agent is built for that loop: index against the request list, map which entity holds which contract, read against the thesis, catch the master-agreement-versus-amendment-versus-side-letter contradiction, and re-run on every upload.

Escalation design is the part most teams underbuild. Confidence thresholds should route to a human queue rather than to a lower-confidence answer, and the queue should show the evidence rather than the conclusion. We covered the pattern in designing approval steps for AI agents; in diligence, any finding crossing the materiality line is queued regardless of model confidence.

A data room is the most confidential corpus either side of a transaction will ever hold, and it is governed by an NDA that names permitted recipients. Uploading it to a hosted review tool that retains files, logs prompts or reserves training rights adds a recipient nobody approved, and that is a disclosure question you will be asked about later.

What that rules in is narrow and workable: a private deployment inside the buyer or adviser network, running against a mirrored copy of the room, with per-workstream access scoping that matches the data room's own permissions, so the tax workstream's agent cannot read the employment folder. No open-web calls at inference time. Research runs against sources the client already licenses, mirrored inside the network. When sizing that box, note the quantization finding above: measure accuracy before you shrink the model to fit the hardware, not after. The trade-offs are in our comparison of cloud versus on-premise AI.

Two regimes attach almost automatically. If the target has EU employees or customers, the room holds personal data and GDPR governs the processing and where it happens. Under the EU AI Act, a reading and extraction tool used for transaction diligence is not typically an Annex III high-risk system, but deployer obligations including AI literacy still attach, and pointing the same pipeline at employment screening changes the classification. To check where a use lands, we built a free classifier for the Act's risk tiers.