Papers › Visual Semantics Allow for Textual Reasoning Better in Scene Text Recognition
Visual Semantics Allow for Textual Reasoning Better in Scene Text Recognition
Yue He, Chen Chen, Jing Zhang, Juhua Liu, Fengxiang He, Chaoyue Wang, Bo Du
Existing Scene Text Recognition (STR) methods typically use a language model to optimize the joint probability of the 1D character sequence predicted by a visual recognition (VR) model, which ignore the 2D spatial context of visual semantics within and between character instances, making them not generalize well to arbitrary shape scene text. To address this issue, we make the first attempt to perform textual reasoning based on visual semantics in this paper. Technically, given the character segmentation maps predicted by a VR model, we construct a subgraph for each instance, where nodes represent the pixels in it and edges are added between nodes based on their spatial similarity. Then, these subgraphs are sequentially connected by their root nodes and merged into a complete graph. Based on this graph, we devise a graph convolutional network for textual reasoning (GTR) by supervising it with a cross-entropy loss. GTR can be easily plugged in representative STR models to improve their performance owing to better textual reasoning. Specifically, we construct our model, namely S-GTR, by paralleling GTR to the language model in a segmentation-based STR baseline, which can effectively exploit the visual-linguistic complementarity via mutual learning. S-GTR sets new state-of-the-art on six challenging STR benchmarks and generalizes well to multi-linguistic datasets. Code is available at https://github.com/adeline-cs/GTR.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Scene Text Recognition | CUTE80 | S-GTR | Accuracy | 94.7 | #12 of 18 | Archive leaderboard | report |
| Scene Text Recognition | ICDAR2013 | S-GTR | Accuracy | 97.8 | #11 of 38 | Archive leaderboard | report |
| Scene Text Recognition | ICDAR2015 | S-GTR | Accuracy | 87.3 | #10 of 27 | Archive leaderboard | report |
| Scene Text Recognition | IIIT5k | S-GTR | Accuracy | 97.5 | #11 of 17 | Archive leaderboard | report |
| Scene Text Recognition | SVT | S-GTR | Accuracy | 95.8 | #13 of 37 | Archive leaderboard | report |
| Scene Text Recognition | SVTP | S-GTR | Accuracy | 90.6 | #12 of 17 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections