{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/when-counting-meets-hmer-counting-aware","title":"When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognition","arxiv_id":"2207.11463","date":"2022-07-23","proceeding":null,"authors":["Bohan Li","Ye Yuan","Dingkang Liang","Xiao Liu","Zhilong Ji","Jinfeng Bai","Wenyu Liu","Xiang Bai"],"abstract":"Recently, most handwritten mathematical expression recognition (HMER) methods adopt the encoder-decoder networks, which directly predict the markup sequences from formula images with the attention mechanism. However, such methods may fail to accurately read formulas with complicated structure or generate long markup sequences, as the attention results are often inaccurate due to the large variance of writing styles or spatial layouts. To alleviate this problem, we propose an unconventional network for HMER named Counting-Aware Network (CAN), which jointly optimizes two tasks: HMER and symbol counting. Specifically, we design a weakly-supervised counting module that can predict the number of each symbol class without the symbol-level position annotations, and then plug it into a typical attention-based encoder-decoder model for HMER. Experiments on the benchmark datasets for HMER validate that both joint optimization and counting results are beneficial for correcting the prediction errors of encoder-decoder models, and CAN consistently outperforms the state-of-the-art methods. In particular, compared with an encoder-decoder model for HMER, the extra time cost caused by the proposed counting module is marginal. The source code is available at https://github.com/LBH1024/CAN.","url_abs":"https://arxiv.org/abs/2207.11463v1","url_pdf":"https://arxiv.org/pdf/2207.11463v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"when-counting-meets-hmer-counting-aware","repo_url":"https://github.com/lbh1024/can","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"when-counting-meets-hmer-counting-aware","repo_url":"https://github.com/qingzhenduyu/ical","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"when-counting-meets-hmer-counting-aware","repo_url":"https://github.com/2024-MindSpore-1/Code3/tree/main/CAN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"handwritten-mathmatical-expression","task_name":"Handwritten Mathmatical Expression Recognition"},{"task_slug":"optical-character-recognition","task_name":"Optical Character Recognition (OCR)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/handwritten-mathmatical-expression","task":"Handwritten Mathmatical Expression Recognition","dataset":"CROHME 2014","model":"CAN-ABM","rank_in_archive_order":8,"of":17,"metrics":{"ExpRate":"57.26"},"uses_additional_data":false},{"leaderboard":"/sota/handwritten-mathmatical-expression","task":"Handwritten Mathmatical Expression Recognition","dataset":"CROHME 2014","model":"CAN-DWAP","rank_in_archive_order":9,"of":17,"metrics":{"ExpRate":"57.00"},"uses_additional_data":false},{"leaderboard":"/sota/handwritten-mathmatical-expression-1","task":"Handwritten Mathmatical Expression Recognition","dataset":"CROHME 2016","model":"CAN-ABM","rank_in_archive_order":7,"of":16,"metrics":{"ExpRate":"56.15"},"uses_additional_data":false},{"leaderboard":"/sota/handwritten-mathmatical-expression-1","task":"Handwritten Mathmatical Expression Recognition","dataset":"CROHME 2016","model":"CAN-DWAP","rank_in_archive_order":8,"of":16,"metrics":{"ExpRate":"56.06"},"uses_additional_data":false},{"leaderboard":"/sota/handwritten-mathmatical-expression-2","task":"Handwritten Mathmatical Expression Recognition","dataset":"CROHME 2019","model":"CAN-ABM","rank_in_archive_order":8,"of":14,"metrics":{"ExpRate":"55.96"},"uses_additional_data":false},{"leaderboard":"/sota/handwritten-mathmatical-expression-2","task":"Handwritten Mathmatical Expression Recognition","dataset":"CROHME 2019","model":"CAN-DWAP","rank_in_archive_order":9,"of":14,"metrics":{"ExpRate":"54.88"},"uses_additional_data":false},{"leaderboard":"/sota/handwritten-mathmatical-expression-3","task":"Handwritten Mathmatical Expression Recognition","dataset":"HME100K","model":"CAN-ABM","rank_in_archive_order":6,"of":12,"metrics":{"ExpRate":"68.09"},"uses_additional_data":false},{"leaderboard":"/sota/handwritten-mathmatical-expression-3","task":"Handwritten Mathmatical Expression Recognition","dataset":"HME100K","model":"CAN-DWAP","rank_in_archive_order":7,"of":12,"metrics":{"ExpRate":"67.31"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2207.11463","atlas_url":"https://app.syntology.ai/?focus=2207.11463","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.11463"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lbh1024/can","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/2024-MindSpore-1/Code3/tree/main/CAN","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qingzhenduyu/ical","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/LBH1024/CAN","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"unverified":1},"by_repo_kind":{"official":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"02d543693c5cbc61","entry":"cal_score","repo":"LBH1024/CAN","repo_kind":"official","path":"utils.py","file_url":"https://github.com/LBH1024/CAN/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"02d543693c5cbc61"}},{"code_sha256_prefix":"9de8449fe1430ac2","entry":"collate_fn","repo":"LBH1024/CAN","repo_kind":"official","path":"dataset.py","file_url":"https://github.com/LBH1024/CAN/blob/HEAD/dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9de8449fe1430ac2"}},{"code_sha256_prefix":"d60c6502d12a92c0","entry":"gen_counting_label","repo":"LBH1024/CAN","repo_kind":"official","path":"counting_utils.py","file_url":"https://github.com/LBH1024/CAN/blob/HEAD/counting_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d60c6502d12a92c0"}},{"code_sha256_prefix":"f00a1fbee53a83e2","entry":"get_crohme_dataset","repo":"LBH1024/CAN","repo_kind":"official","path":"dataset.py","file_url":"https://github.com/LBH1024/CAN/blob/HEAD/dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f00a1fbee53a83e2"}},{"code_sha256_prefix":"6f75ee616b44ea28","entry":"load_config","repo":"LBH1024/CAN","repo_kind":"official","path":"utils.py","file_url":"https://github.com/LBH1024/CAN/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6f75ee616b44ea28"}},{"code_sha256_prefix":"beb2b433b3cefd2f","entry":"save_checkpoint","repo":"LBH1024/CAN","repo_kind":"official","path":"utils.py","file_url":"https://github.com/LBH1024/CAN/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"beb2b433b3cefd2f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}