VESUVIUS CHALLENGE / PUBLIC FINDINGS
Our published results concern engineering components: image filtering, letter hypotheses and workflow checks. A higher score on a defined task does not establish a new reading or success on a previously unseen scroll.
The released Inkfilter face-mix recipe was compared with a selected alpha-compositing baseline on 56 confirmation windows. These were new windows from the same 28 segments and seven scrolls used in development, not wholly unseen scrolls. The recipe operates around 4.8 micrometres per pixel.
| Method | Mean ROC-AUC |
|---|---|
| Frozen face-mix filter | 0.7357 |
| Selected alpha-compositing baseline | 0.7042 |
| Reversed-depth face-mix control | 0.6276 |
ROC-AUC measures how well scores rank labeled ink above labeled non-ink; it is not the fraction of letters read. The mean favors face-mix in this comparison, but alpha alone wins on three scrolls and the reverse filter wins on one. Those negative results remain part of the evidence.
What this supports: a compact, nonlearned baseline worth comparing with other fixed filters on similar data. What it does not support: superiority to trained readers, reliability on a new scroll or resolution, or readable text from these outputs.
The public evidence includes the window list, saved scalar scores, intervals, controls and analysis definitions. Its aggregation script recomputes the summary from those saved scores. It does not regenerate the original maps or full baseline bank from raw CT. The new local-file adapter was tested synthetically; that is separate from the earlier numerical study.
The public Scroll Text Reader comparison evaluated two released Greek models on PHerc. 1667. With line positions and letter height supplied, the letters-v2-real model had lower free-ends character error on the tested ink-map groups than letters-v2-ccby.
| Released model | Free-ends character error |
|---|---|
| letters-v2-real | 0.376 |
| letters-v2-ccby | 0.462 |
The paired difference, ccby − real, was +0.085, with a 95% bootstrap interval of +0.017 to +0.155 (computed before rounding). The test scored 84 unanimous groups containing 117 letters. The references were rebuilt blind machine transcriptions of the public drawings, not a papyrologist’s independent reading.
When the reader processed each segment’s whole labeled region without supplied lines or scale, the comparison was unresolved: the interval included zero. The public model cards also report synthetic pages, PHerc. 172 diagnostic checks and Hell-Char validation. Their checkpoint selection and dataset exposure differ; these numbers are not an untouched, unseen-scroll generalization result. The supplied-line result is one of six predefined scroll comparisons, with no adjustment for multiple comparisons; its interval is not a family-wide confirmation. Free-ends error is a particular alignment score, not a percentage of a scroll deciphered.
The practical choice: use letters-v2-real for the documented supplied-line research profile when its noncommercial license fits; the permissively licensed letters-v2-ccby remains the automatic-layout and demo default. Neither choice turns hypotheses into a verified transcription. Tesseract’s supported languages are a backend capability, not measured scroll accuracy.
The reader’s public adversarial test arranged random pen strokes into rows. All eight tested pages received a “strong” status despite not containing letters. The status detects evidence relative to its controls, not semantic truth. The reader documentation explains the control behavior and examples.
Depth reversal keeps much of the sheet’s structure while changing its depth order. A forward filter should outperform a matched reverse control when its claimed advantage depends on the ink-side arrangement. Failure is a warning; success alone does not identify letters.
Starter uses a known synthetic answer. The reader demo tests clean synthetic text and noise. AutoPipeline’s demo checks stage orchestration with stand-ins. These are useful installation and logic checks, distinct from real-data accuracy.
A model cannot repair every crossed layer, incorrect axis or inappropriate pixel scale. Compare like with like: the same crop, registration, score mask and physical resolution. Keep unsuccessful cases beside successful ones.
Questions, corrections and small reproductions belong in the public issue tracker when they contain only material you are permitted to share. Link the release, exact command and error; use synthetic inputs for a software bug whenever possible. Do not attach private scroll maps or candidate readings.
Public sources reviewed 18 October 2026. Hendrick Research is an independent contributor, not the organizer of the Vesuvius Challenge. Software, models and datasets have separate licenses. Public source and corrections. Screenshot sources and hashes.