Papers › Uncovering the Handwritten Text in the Margins: End-to-end Handwritten Text Detection...

Uncovering the Handwritten Text in the Margins: End-to-end Handwritten Text Detection and Recognition

10 Mar 2023arXiv:2303.05929archive 2025-07-28

Liang Cheng, Jonas Frankemölle, Adam Axelsson, Ekta Vats

The pressing need for digitization of historical documents has led to a strong interest in designing computerised image processing methods for automatic handwritten text recognition. However, not much attention has been paid on studying the handwritten text written in the margins, i.e. marginalia, that also forms an important source of information. Nevertheless, training an accurate and robust recognition system for marginalia calls for data-efficient approaches due to the unavailability of sufficient amounts of annotated multi-writer texts. Therefore, this work presents an end-to-end framework for automatic detection and recognition of handwritten marginalia, and leverages data augmentation and transfer learning to overcome training data scarcity. The detection phase involves investigation of R-CNN and Faster R-CNN networks. The recognition phase includes an attention-based sequence-to-sequence model, with ResNet feature extraction, bidirectional LSTM-based sequence modeling, and attention-based prediction of marginalia. The effectiveness of the proposed framework has been empirically evaluated on the data from early book collections found in the Uppsala University Library in Sweden. Source code and pre-trained models are available at Github.

Paper PDF Code

Code

adamaxelsson/project-marginalia officialmentioned in papermentioned on GitHubpytorch report

ektavats/project-marginalia officialmentioned in papermentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Data Augmentation Handwritten Text Recognition Text Detection Transfer Learning

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

1x1 Convolution Average Pooling Batch Normalization Bottleneck Residual Block Convolution Faster R-CNN Global Average Pooling Kaiming Initialization Max Pooling RPN ReLU Residual Block Residual Connection RoIPool Softmax

1 archive method tag without a method page not shown.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections