{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/syntax-aware-network-for-handwritten","title":"Syntax-Aware Network for Handwritten Mathematical Expression Recognition","arxiv_id":"2203.01601","date":"2022-03-03","proceeding":"CVPR 2022 1","authors":["Ye Yuan","Xiao Liu","Wondimu Dikubab","Hui Liu","Zhilong Ji","Zhongqin Wu","Xiang Bai"],"abstract":"Handwritten mathematical expression recognition (HMER) is a challenging task that has many potential applications. Recent methods for HMER have achieved outstanding performance with an encoder-decoder architecture. However, these methods adhere to the paradigm that the prediction is made \"from one character to another\", which inevitably yields prediction errors due to the complicated structures of mathematical expressions or crabbed handwritings. In this paper, we propose a simple and efficient method for HMER, which is the first to incorporate syntax information into an encoder-decoder network. Specifically, we present a set of grammar rules for converting the LaTeX markup sequence of each expression into a parsing tree; then, we model the markup sequence prediction as a tree traverse process with a deep neural network. In this way, the proposed method can effectively describe the syntax context of expressions, alleviating the structure prediction errors of HMER. Experiments on three benchmark datasets demonstrate that our method achieves better recognition performance than prior arts. To further validate the effectiveness of our method, we create a large-scale dataset consisting of 100k handwritten mathematical expression images acquired from ten thousand writers. The source code, new dataset, and pre-trained models of this work will be publicly available.","url_abs":"https://arxiv.org/abs/2203.01601v4","url_pdf":"https://arxiv.org/pdf/2203.01601v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"syntax-aware-network-for-handwritten","repo_url":"https://github.com/tal-tech/san","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"syntax-aware-network-for-handwritten","repo_url":"https://github.com/phymond/hme100k","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"handwritten-mathmatical-expression","task_name":"Handwritten Mathmatical Expression Recognition"},{"task_slug":"prediction","task_name":"Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/handwritten-mathmatical-expression","task":"Handwritten Mathmatical Expression Recognition","dataset":"CROHME 2014","model":"SAN","rank_in_archive_order":11,"of":17,"metrics":{"ExpRate":"56.2"},"uses_additional_data":false},{"leaderboard":"/sota/handwritten-mathmatical-expression-1","task":"Handwritten Mathmatical Expression Recognition","dataset":"CROHME 2016","model":"SAN","rank_in_archive_order":10,"of":16,"metrics":{"ExpRate":"53.6"},"uses_additional_data":false},{"leaderboard":"/sota/handwritten-mathmatical-expression-2","task":"Handwritten Mathmatical Expression Recognition","dataset":"CROHME 2019","model":"SAN","rank_in_archive_order":11,"of":14,"metrics":{"ExpRate":"53.5"},"uses_additional_data":false},{"leaderboard":"/sota/handwritten-mathmatical-expression-3","task":"Handwritten Mathmatical Expression Recognition","dataset":"HME100K","model":"SAN","rank_in_archive_order":8,"of":12,"metrics":{"ExpRate":"67.1"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.01601","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.01601"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tal-tech/san","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/phymond/hme100k","reach":{"status":"ok"}}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"204a7c15bb553044","entry":"Attention","repo":"tal-tech/san","repo_kind":"official","path":"models/Hierarchical_attention/decoder.py","file_url":"https://github.com/tal-tech/san/blob/HEAD/models/Hierarchical_attention/decoder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"204a7c15bb553044"}},{"code_sha256_prefix":"7fb90f2c6bc22a52","entry":"SAN_decoder","repo":"tal-tech/san","repo_kind":"official","path":"models/Hierarchical_attention/decoder.py","file_url":"https://github.com/tal-tech/san/blob/HEAD/models/Hierarchical_attention/decoder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7fb90f2c6bc22a52"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}