{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/posformer-recognizing-complex-handwritten","title":"PosFormer: Recognizing Complex Handwritten Mathematical Expression with Position Forest Transformer","arxiv_id":"2407.07764","date":"2024-07-10","proceeding":null,"authors":["Tongkun Guan","Chengyu Lin","Wei Shen","Xiaokang Yang"],"abstract":"Handwritten Mathematical Expression Recognition (HMER) has wide applications in human-machine interaction scenarios, such as digitized education and automated offices. Recently, sequence-based models with encoder-decoder architectures have been commonly adopted to address this task by directly predicting LaTeX sequences of expression images. However, these methods only implicitly learn the syntax rules provided by LaTeX, which may fail to describe the position and hierarchical relationship between symbols due to complex structural relations and diverse handwriting styles. To overcome this challenge, we propose a position forest transformer (PosFormer) for HMER, which jointly optimizes two tasks: expression recognition and position recognition, to explicitly enable position-aware symbol feature representation learning. Specifically, we first design a position forest that models the mathematical expression as a forest structure and parses the relative position relationships between symbols. Without requiring extra annotations, each symbol is assigned a position identifier in the forest to denote its relative spatial position. Second, we propose an implicit attention correction module to accurately capture attention for HMER in the sequence-based decoder architecture. Extensive experiments validate the superiority of PosFormer, which consistently outperforms the state-of-the-art methods 2.03%/1.22%/2.00%, 1.83%, and 4.62% gains on the single-line CROHME 2014/2016/2019, multi-line M2E, and complex MNE datasets, respectively, with no additional latency or computational cost. Code is available at https://github.com/SJTU-DeepVisionLab/PosFormer.","url_abs":"https://arxiv.org/abs/2407.07764v1","url_pdf":"https://arxiv.org/pdf/2407.07764v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"posformer-recognizing-complex-handwritten","repo_url":"https://github.com/sjtu-deepvisionlab/posformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"handwritten-mathmatical-expression","task_name":"Handwritten Mathmatical Expression Recognition"},{"task_slug":null,"task_name":"Position"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/handwritten-mathmatical-expression","task":"Handwritten Mathmatical Expression Recognition","dataset":"CROHME 2014","model":"PosFormer","rank_in_archive_order":6,"of":17,"metrics":{"ExpRate":"60.45"},"uses_additional_data":false},{"leaderboard":"/sota/handwritten-mathmatical-expression-1","task":"Handwritten Mathmatical Expression Recognition","dataset":"CROHME 2016","model":"PosFormer","rank_in_archive_order":2,"of":16,"metrics":{"ExpRate":"60.94"},"uses_additional_data":false},{"leaderboard":"/sota/handwritten-mathmatical-expression-2","task":"Handwritten Mathmatical Expression Recognition","dataset":"CROHME 2019","model":"PosFormer","rank_in_archive_order":2,"of":14,"metrics":{"ExpRate":"62.22"},"uses_additional_data":false},{"leaderboard":"/sota/handwritten-mathmatical-expression-3","task":"Handwritten Mathmatical Expression Recognition","dataset":"HME100K","model":"PosFormer","rank_in_archive_order":1,"of":12,"metrics":{"ExpRate":"69.51"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.07764","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.07764"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sjtu-deepvisionlab/posformer","reach":{"status":"ok"}}],"summary":{"ran":8,"unverified":1},"by_repo_kind":{"official":{"samples":9,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"b5f47ef41364e05e","entry":"data_iterator","repo":"sjtu-deepvisionlab/posformer","repo_kind":"official","path":"m2e_pkg/datamodule.py","file_url":"https://github.com/sjtu-deepvisionlab/posformer/blob/HEAD/m2e_pkg/datamodule.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b5f47ef41364e05e"}},{"code_sha256_prefix":"62322ab0e7d1e9f9","entry":"data_iterator","repo":"sjtu-deepvisionlab/posformer","repo_kind":"official","path":"Pos_Former/datamodule/datamodule.py","file_url":"https://github.com/sjtu-deepvisionlab/posformer/blob/HEAD/Pos_Former/datamodule/datamodule.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"62322ab0e7d1e9f9"}},{"code_sha256_prefix":"3d952d11c1bd842b","entry":"extract_data","repo":"sjtu-deepvisionlab/posformer","repo_kind":"official","path":"m2e_pkg/datamodule.py","file_url":"https://github.com/sjtu-deepvisionlab/posformer/blob/HEAD/m2e_pkg/datamodule.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3d952d11c1bd842b"}},{"code_sha256_prefix":"1fccc851efffb7be","entry":"extract_data","repo":"sjtu-deepvisionlab/posformer","repo_kind":"official","path":"Pos_Former/datamodule/datamodule.py","file_url":"https://github.com/sjtu-deepvisionlab/posformer/blob/HEAD/Pos_Former/datamodule/datamodule.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1fccc851efffb7be"}},{"code_sha256_prefix":"ed1ff4e28224104b","entry":"find_end_bigbracket","repo":"sjtu-deepvisionlab/posformer","repo_kind":"official","path":"Pos_Former/datamodule/label_make_muti.py","file_url":"https://github.com/sjtu-deepvisionlab/posformer/blob/HEAD/Pos_Former/datamodule/label_make_muti.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ed1ff4e28224104b"}},{"code_sha256_prefix":"6552f1752335a8fd","entry":"find_end_midbracket","repo":"sjtu-deepvisionlab/posformer","repo_kind":"official","path":"Pos_Former/datamodule/label_make_muti.py","file_url":"https://github.com/sjtu-deepvisionlab/posformer/blob/HEAD/Pos_Former/datamodule/label_make_muti.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6552f1752335a8fd"}},{"code_sha256_prefix":"f50f3c4d7aec4787","entry":"helper","repo":"sjtu-deepvisionlab/posformer","repo_kind":"official","path":"Pos_Former/datamodule/label_make_muti.py","file_url":"https://github.com/sjtu-deepvisionlab/posformer/blob/HEAD/Pos_Former/datamodule/label_make_muti.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f50f3c4d7aec4787"}},{"code_sha256_prefix":"e06d2f5f2824ecc6","entry":"rotate_every_two","repo":"sjtu-deepvisionlab/posformer","repo_kind":"official","path":"Pos_Former/model/pos_enc.py","file_url":"https://github.com/sjtu-deepvisionlab/posformer/blob/HEAD/Pos_Former/model/pos_enc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e06d2f5f2824ecc6"}},{"code_sha256_prefix":"9f2e3298b5be99e0","entry":"collate_fn","repo":"sjtu-deepvisionlab/posformer","repo_kind":"official","path":"m2e_pkg/datamodule.py","file_url":"https://github.com/sjtu-deepvisionlab/posformer/blob/HEAD/m2e_pkg/datamodule.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9f2e3298b5be99e0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}