What we measured, and how.
These are the results we have measured. Each one names the population it was measured over, the conditions it was measured under and a receipt you can download and check yourself.
MEASUREDRecovery changes the outcome
What this means for you: on a page that is hard to read, the recovery step is the difference between getting the document's content and getting an empty page.
On olmOCR-Bench, scored by the benchmark's own evaluator at revision cfa88c1e, the same pipeline scored 80.6 with the recovery step and 53.7 with only that step switched off — a gap of 26.9 points, 95% confidence intervals 79.62–81.57 and 52.62–54.93, which do not overlap. Model, evaluator revision, corpus, source manifest, test set and settings were identical; the only difference was whether the documents recovery delivered carried their content. Measured 2026-08-08 over 1,403 documents and 8,413 checks. One category, headers and footers, scores higher without recovery, because a check that a phrase is absent passes trivially on an empty page — the no-recovery figure is generous rather than harsh.
Receipt R-01 · recovery counterfactual on olmOCR-Bench · 2026-08-08 · Download the receipt
sha256 1f5b6220c1fa569e8e33530d933a16eb7ad6b856c56e22f1744f8fa96efe33e0
MEASUREDResults by document type
What this means for you: results depend on the kind of document, so the per-type breakdown in the receipt is a better guide than the overall average.
On olmOCR-Bench, scored by the benchmark's own evaluator at revision cfa88c1e over 1,403 documents and 8,413 checks, the overall score was 80.6. The receipt lists the result for every document type in the set. Measured 2026-08-04. This is the with-recovery arm of R-01, a research measurement on that corpus and that build, not this service's live rate.
Receipt R-03 · official olmOCR-Bench evaluation, by document type · 2026-08-04 · Download the receipt
sha256 4d75ae96e70c7c1c425b991df0f1c356ec6393e73914aaed27f5f54b0048a559
MEASUREDCompilation refuses more than it emits, sometimes
What this means for you: when a document points at something that was not supplied with it, the compiler refuses that document rather than emitting a world with a dead link in it.
Of a thousand documents offered, 596 compiled and 404 were refused, every one for a link the compiler could not resolve — most often a referenced figure asset that had not been supplied alongside the markdown. A vault with a broken link is not emitted, by design. Measured 2026-08-08, on that corpus and that build: a historical research measurement, not this service's live refusal rate, which is not published.
Receipt R-02 · knowledge-compilation properties on a thousand-document corpus · 2026-08-08 · Download the receipt
sha256 936b859c484fb54a8bdff3175d89d2fd47d695d48ec93b99fcfd93ac53ee2e25
Every receipt above is bound by sha256, and every one of them is published here. Download the file, hash it yourself, and compare it with the digest printed beside it — shasum -a 256 <file> on macOS or Linux, Get-FileHash <file> on Windows. A receipt that is named on this page and not published beside it says so in its own row.
No number here is placed beside a competitor’s result as though it were reproduced under the same conditions. Comparative work is published only after a baseline is reproduced on a frozen configuration, with the raw outputs and the failures included.
Also in the trust case: Evidence · Benchmarks · Reproducibility · Research · Trust