{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/geoqa-a-geometric-question-answering","title":"GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning","arxiv_id":"2105.14517","date":"2021-05-30","proceeding":"Findings (ACL) 2021 8","authors":["Jiaqi Chen","Jianheng Tang","Jinghui Qin","Xiaodan Liang","Lingbo Liu","Eric P. Xing","Liang Lin"],"abstract":"Automatic math problem solving has recently attracted increasing attention as a long-standing AI benchmark. In this paper, we focus on solving geometric problems, which requires a comprehensive understanding of textual descriptions, visual diagrams, and theorem knowledge. However, the existing methods were highly dependent on handcraft rules and were merely evaluated on small-scale datasets. Therefore, we propose a Geometric Question Answering dataset GeoQA, containing 4,998 geometric problems with corresponding annotated programs, which illustrate the solving process of the given problems. Compared with another publicly available dataset GeoS, GeoQA is 25 times larger, in which the program annotations can provide a practical testbed for future research on explicit and explainable numerical reasoning. Moreover, we introduce a Neural Geometric Solver (NGS) to address geometric problems by comprehensively parsing multimodal information and generating interpretable programs. We further add multiple self-supervised auxiliary tasks on NGS to enhance cross-modal semantic representation. Extensive experiments on GeoQA validate the effectiveness of our proposed NGS and auxiliary tasks. However, the results are still significantly lower than human performance, which leaves large room for future research. Our benchmark and code are released at https://github.com/chen-judge/GeoQA .","url_abs":"https://arxiv.org/abs/2105.14517v3","url_pdf":"https://arxiv.org/pdf/2105.14517v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"geoqa-a-geometric-question-answering","repo_url":"https://github.com/chen-judge/GeoQA","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"math","task_name":"Math"},{"task_slug":"mathematical-reasoning","task_name":"Mathematical Reasoning"},{"task_slug":"question-answering","task_name":"Question Answering"}],"methods":[],"datasets_introduced":[{"slug":"geoqa","name":"GeoQA","full_name":"Geometric Question Answering"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/mathematical-reasoning-on-pgps9k","task":"Mathematical Reasoning","dataset":"PGPS9K","model":"NGS","rank_in_archive_order":6,"of":6,"metrics":{"Completion accuracy":"34.1"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2105.14517","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2105.14517"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chen-judge/GeoQA","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/chenjudge/GeoQA","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}}],"summary":{"unverified":5},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"cc10a482843ff5dd","entry":"FFN","repo":"chen-judge/GeoQA","repo_kind":"official","path":"mcan.py","file_url":"https://github.com/chen-judge/GeoQA/blob/HEAD/mcan.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cc10a482843ff5dd"}},{"code_sha256_prefix":"f39d71bbae7705a3","entry":"MCA_ED","repo":"chen-judge/GeoQA","repo_kind":"official","path":"mcan.py","file_url":"https://github.com/chen-judge/GeoQA/blob/HEAD/mcan.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f39d71bbae7705a3"}},{"code_sha256_prefix":"983500f97e817249","entry":"MHAtt","repo":"chen-judge/GeoQA","repo_kind":"official","path":"mcan.py","file_url":"https://github.com/chen-judge/GeoQA/blob/HEAD/mcan.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"983500f97e817249"}},{"code_sha256_prefix":"f809f0badbbf83e5","entry":"SA","repo":"chen-judge/GeoQA","repo_kind":"official","path":"mcan.py","file_url":"https://github.com/chen-judge/GeoQA/blob/HEAD/mcan.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f809f0badbbf83e5"}},{"code_sha256_prefix":"20645eed8c0870c3","entry":"SGA","repo":"chen-judge/GeoQA","repo_kind":"official","path":"mcan.py","file_url":"https://github.com/chen-judge/GeoQA/blob/HEAD/mcan.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"20645eed8c0870c3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}