Papers › Handwritten Text Recognition from Crowdsourced Annotations

Handwritten Text Recognition from Crowdsourced Annotations

19 Jun 2023International Workshop on Historical Document Imaging and Processing 2023 6arXiv:2306.10878archive 2025-07-28

Solène Tarride, Tristan Faine, Mélodie Boillet, Harold Mouchère, Christopher Kermorvant

In this paper, we explore different ways of training a model for handwritten text recognition when multiple imperfect or noisy transcriptions are available. We consider various training configurations, such as selecting a single transcription, retaining all transcriptions, or computing an aggregated transcription from all available annotations. In addition, we evaluate the impact of quality-based data selection, where samples with low agreement are removed from the training set. Our experiments are carried out on municipal registers of the city of Belfort (France) written between 1790 and 1946. % results The results show that computing a consensus transcription or training on multiple transcriptions are good alternatives. However, selecting training samples based on the degree of agreement between annotators introduces a bias in the training data and does not improve the results. Our dataset is publicly available on Zenodo: https://zenodo.org/record/8041668.

PaperPDFConference PDF

Code

No code repository is listed for this paper in the archive or in Syntology's graph.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Handwritten Text Recognition

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Handwritten Text Recognition Belfort PyLaia (all transcriptions + agreement-based split) CER (%) 4.34 #1 of 4 Archive leaderboard report
Handwritten Text Recognition Belfort PyLaia (all transcriptions + agreement-based split) WER (%) 15.14 #1 of 4 Archive leaderboard report
Handwritten Text Recognition Belfort PyLaia (rover consensus + agreement-based split) CER (%) 4.95 #2 of 4 Archive leaderboard report
Handwritten Text Recognition Belfort PyLaia (rover consensus + agreement-based split) WER (%) 17.08 #2 of 4 Archive leaderboard report
Handwritten Text Recognition Belfort PyLaia (human transcriptions + agreement-based split) CER (%) 5.57 #3 of 4 Archive leaderboard report
Handwritten Text Recognition Belfort PyLaia (human transcriptions + agreement-based split) WER (%) 19.12 #3 of 4 Archive leaderboard report
Handwritten Text Recognition Belfort PyLaia (human transcriptions + random split) CER (%) 10.54 #4 of 4 Archive leaderboard report
Handwritten Text Recognition Belfort PyLaia (human transcriptions + random split) WER (%) 28.11 #4 of 4 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections