What we measured, and how.
This is the record of what has actually been measured, and what has not. Every result carries the state of its evidence: a measurement is a number we produced and can describe the conditions for, open is code that passes its tests without that making a threshold right, and not supported is a hypothesis we tested, that failed, and that we did not ship anyway.
MEASUREDRecovery changes the outcome
What this means for you: on a page that is hard to read, the recovery step is the difference between getting the document's content and getting an empty page.
On olmOCR-Bench, scored by the benchmark's own evaluator at revision cfa88c1e, the same pipeline scored 80.6 with the recovery lane and 53.7 with only that lane switched off — a gap of 26.9 points, 95% confidence intervals 79.62–81.57 and 52.62–54.93, which do not overlap. Model, evaluator revision, corpus, source manifest, test set and settings were identical; the only difference was whether the documents recovery delivered carried their content. Measured 2026-08-08 over 1,403 documents and 8,413 checks. One category, headers and footers, scores higher without recovery, because a check that a phrase is absent passes trivially on an empty page — the no-recovery figure is generous rather than harsh.
Receipt R-01 · recovery counterfactual on olmOCR-Bench · 2026-08-08 · Download the receipt
sha256 1f5b6220c1fa569e8e33530d933a16eb7ad6b856c56e22f1744f8fa96efe33e0
MEASUREDCompilation refuses more than it emits, sometimes
What this means for you: when a document points at something that was not supplied with it, the compiler refuses that document rather than emitting a world with a dead link in it.
Of a thousand documents offered, 596 compiled and 404 were refused, every one for a link the compiler could not resolve — most often a referenced figure asset that had not been supplied alongside the markdown. A vault with a broken link is not emitted, by design. Measured 2026-08-08, on that corpus and that build: a historical research measurement, not this service's live refusal rate, which is not published.
Receipt R-02 · knowledge-compilation properties on a thousand-document corpus · 2026-08-08 · Download the receipt
sha256 936b859c484fb54a8bdff3175d89d2fd47d695d48ec93b99fcfd93ac53ee2e25
NOT SUPPORTEDBlind quality detection failed
What this means for you: nothing here ranks your documents by a quality score, because the score we tested did not beat sorting by length.
We tested whether prediction-only signals could pick the worst documents without ground truth. They could not beat ranking by length alone. Published as unsupported, and not shipped as a feature.
OPENThresholds are set by judgement, not calibration
What this means for you: the cut-offs that decide when the system refuses or merges are set by judgement rather than by a measurement, so ask before you depend on one.
Tests show the code does what its author intended. They do not show a threshold is right. No threshold here is presented as a measured result.
OPENRebuilding only what changed
What this means for you: the landing-page demonstration runs on sample data, and it is not a capability TAVONEL gives you today.
The landing demonstration follows a dependency path on declared sample data. It is not a shipped capability.
Every receipt above is bound by sha256, and every one of them is published here. Download the file, hash it yourself, and compare it with the digest printed beside it — shasum -a 256 <file> on macOS or Linux, Get-FileHash <file> on Windows. A receipt that is named on this page and not published beside it says so in its own row.
No number here is placed beside a competitor’s result as though it were reproduced under the same conditions. Comparative work is published only after a baseline is reproduced on a frozen configuration, with the raw outputs and the failures included.
Also in the trust case: Evidence · Benchmarks · Reproducibility · Research · Trust