/evidence

Anyone can put a document in a search box.

Ingestion audit and sample investigation

The value is in what survives the trip. An answer that loses the table it depended on, misattributes a clause, or cannot say which page it came from is worse than no answer — it is an answer someone will act on. So we publish what survived, what did not, and how we counted.

Why these documents and not a customer's. Recall cannot be scored against real documents, because nobody knows the true answer in them. The four files below were built to a published answer key, so every claim on this page can be checked and every figure reproduced from seed 20260803. A vendor quoting accuracy on documents with no answer key is quoting an opinion. Figures from customer deployments are shared under NDA, against your own material.

Ingestion audit

Measured on the ServAI benchmark corpus · FMSC-Model-Ed4.pdf · PMSA-Service-Agreement.pdf · MS-114-Routine-Inspection.pdf · Q2-Q3-Maintenance-Register.xlsx · 140 pages · seed 20260803

140Pages ingested — 0 image-only, read as images not skipped

82Tables preserved as tables — 82 with headers intact (100%)

58Figures kept — 58 matched to a caption (100%)

517Sections resolved, numbering to 3 levels deep

Self-check · fixed seed 100.0%

60 of 60 sections re-located

Sections were sampled at random, the source reopened, and each one re-found on the page the index recorded. A section that cannot be re-found is a failure and is named, not dropped. Same seed, same result — a customer can rerun it.

What did not survive 0

Parse failures, named

Cross-references resolved: 1,242 of 1,389 (89.4%) — a reference pointing at a clause the index does not hold is counted as unresolved rather than quietly dropped. Ingested in 28.8s.

Why the number is not 100%. Publishing four counts with no denominator proves nothing. 89.4% of cross-references resolved is a figure you can argue with, which is the point of putting it here.

A worked investigation

Reconcile Q2–Q3 maintenance billing against the service agreement, and confirm inspection intervals meet the code. Run against the same four files: the model code (FMSC-Model-Ed4.pdf), the service agreement (PMSA-Service-Agreement.pdf), method statement MS-114 and a 245-row maintenance register.

Tokens (est.)98,085

Rows analysed245

Gates passed10 / 13

Gates fired2

Returned1

Run time0.2s

Benchmark corpus · every figure computed from the files, none typed by hand

Of AED 284,456 billed across the period, AED 12,118 is not payable on the terms of the agreement.

Category	Lines	Value (AED)	Grounding	Status
Billed above contracted rate	19	3,436	Contract Table 7.1 · p.4 vs register	Recoverable
Visit invoiced twice	5	6,980	Clause 11.1 · register duplicates	Not payable
Completed visit without evidence (15% deduction)	9	1,702	Clause 11.1 read with 12.1	Deductible
Inspection interval exceeded the code maximum	13	—	Code Table 3.1 · p.30	Non-compliance

Gate 09 returned this report once. The first draft rounded each line before summing and reported 1.23% of billing. Recomputed from the rows, the true figure is 1.21%. The draft was not annotated and passed along — it was sent back and rewritten. The reader never saw the wrong number; the audit log did.

Gate 10 fired — two documents disagree

The method statement in use permits a longer inspection interval than the code allows for the same asset classes. Both are cited. The agent did not pick one.

Asset	Code	Source	Method statement	Source
AC-01	60 days	Code · Table 3.1 · p.30	90 days	MS-114 · Table 4.1 · p.2
AC-03	60 days	Code · Table 3.1 · p.30	90 days	MS-114 · Table 4.1 · p.2

Gate 12 fired — question for you. The record contains no reason for the longer interval, and no variation under Clause 10.2 was found. An approved variation that was never filed and a method statement written against a superseded edition carry very different consequences. The agent will not choose between them. Who approved MS-114 Table 4.1, and against which edition?

Scored against ground truth

The corpus carries a known set of planted anomalies written to an answer key the investigation never opens. Comparing afterwards is what makes this falsifiable rather than illustrative.

Category	Planted	Found	False +	Recall
Overcharged lines	18	18	1	100.0%
Duplicate visits	5	5	0	100.0%
Interval breaches	13	13	0	100.0%
Missing evidence	8	8	1	100.0%

Reading the false positives. Lines were flagged that the generator did not plant: they are duplicate rows that inherited the properties of the original they copy. A duplicate of an overcharged visit is itself overcharged, so flagging it is arguably correct — and the scorer counts it against us anyway. That is the right way round. A scorer that forgives its own edge cases measures nothing.

What this run does not establish. The corpus is generated and internally consistent, so recall here is a ceiling, not a forecast — real documents carry inconsistent numbering, scanned pages and clauses that contradict themselves. Interval breaches are computed from the register's own gap column, so a visit never recorded would not appear. Nothing was billed, credited or disputed: the run produced a draft and stopped.

ServAI benchmark corpus · seed 20260803 · reproducible on request.
The four files are generated reference documents built to a published answer key, not customer material and not a real code or agreement. Every figure on this page is computed from those files and is not a measurement of a customer deployment. We will run the same audit against your own documents under NDA.

Bring one document set and one workflow. We will run this against your material and show you the same page with your numbers on it.

Talk to Us →