{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/goedel-prover-a-frontier-model-for-open","title":"Goedel-Prover: A Frontier Model for Open-Source Automated Theorem Proving","arxiv_id":"2502.07640","date":"2025-02-11","proceeding":null,"authors":["Yong Lin","Shange Tang","Bohan Lyu","Jiayun Wu","Hongzhou Lin","Kaiyu Yang","Jia Li","Mengzhou Xia","Danqi Chen","Sanjeev Arora","Chi Jin"],"abstract":"We introduce Goedel-Prover, an open-source large language model (LLM) that achieves the state-of-the-art (SOTA) performance in automated formal proof generation for mathematical problems. The key challenge in this field is the scarcity of formalized math statements and proofs, which we tackle in the following ways. We train statement formalizers to translate the natural language math problems from Numina into formal language (Lean 4), creating a dataset of 1.64 million formal statements. LLMs are used to check that the formal statements accurately preserve the content of the original natural language problems. We then iteratively build a large dataset of formal proofs by training a series of provers. Each prover succeeds in proving many statements that the previous ones could not, and these new proofs are added to the training set for the next prover. The final prover outperforms all existing open-source models in whole-proof generation. On the miniF2F benchmark, it achieves a 57.6% success rate (Pass@32), exceeding the previous best open-source model by 7.6%. On PutnamBench, Goedel-Prover successfully solves 7 problems (Pass@512), ranking first on the leaderboard. Furthermore, it generates 29.7K formal proofs for Lean Workbook problems, nearly doubling the 15.7K produced by earlier works.","url_abs":"https://arxiv.org/abs/2502.07640v1","url_pdf":"https://arxiv.org/pdf/2502.07640v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"goedel-prover-a-frontier-model-for-open","repo_url":"https://github.com/Goedel-LM/Goedel-Prover","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"automated-theorem-proving","task_name":"Automated Theorem Proving"},{"task_slug":"large-language-model","task_name":"Large Language Model"},{"task_slug":"math","task_name":"Math"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2502.07640","atlas_url":"https://app.syntology.ai/?focus=2502.07640","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.07640"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Goedel-LM/Goedel-Prover","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":6},"by_repo_kind":{"official":{"samples":7,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e06460509fb60d43","entry":"extrac_code","repo":"Goedel-LM/Goedel-Prover","repo_kind":"official","path":"eval/step1_inference.py","file_url":"https://github.com/Goedel-LM/Goedel-Prover/blob/HEAD/eval/step1_inference.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e06460509fb60d43"}},{"code_sha256_prefix":"77994fb797e63cb3","entry":"cot_prompt","repo":"Goedel-LM/Goedel-Prover","repo_kind":"official","path":"prover/utils.py","file_url":"https://github.com/Goedel-LM/Goedel-Prover/blob/HEAD/prover/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"77994fb797e63cb3"}},{"code_sha256_prefix":"4222bbd6373963f5","entry":"extract_positions","repo":"Goedel-LM/Goedel-Prover","repo_kind":"official","path":"prover/lean/ast_parser.py","file_url":"https://github.com/Goedel-LM/Goedel-Prover/blob/HEAD/prover/lean/ast_parser.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4222bbd6373963f5"}},{"code_sha256_prefix":"20ac111c42889614","entry":"extract_vals","repo":"Goedel-LM/Goedel-Prover","repo_kind":"official","path":"prover/lean/ast_parser.py","file_url":"https://github.com/Goedel-LM/Goedel-Prover/blob/HEAD/prover/lean/ast_parser.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"20ac111c42889614"}},{"code_sha256_prefix":"23c78fbc0be837a2","entry":"non_cot_few_shot_prompt","repo":"Goedel-LM/Goedel-Prover","repo_kind":"official","path":"prover/utils.py","file_url":"https://github.com/Goedel-LM/Goedel-Prover/blob/HEAD/prover/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"23c78fbc0be837a2"}},{"code_sha256_prefix":"e4ed28dc52c5179d","entry":"non_cot_prompt","repo":"Goedel-LM/Goedel-Prover","repo_kind":"official","path":"prover/utils.py","file_url":"https://github.com/Goedel-LM/Goedel-Prover/blob/HEAD/prover/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e4ed28dc52c5179d"}},{"code_sha256_prefix":"789c40fc9b050fa1","entry":"process_lean_file","repo":"Goedel-LM/Goedel-Prover","repo_kind":"official","path":"prover/lean/ast_parser.py","file_url":"https://github.com/Goedel-LM/Goedel-Prover/blob/HEAD/prover/lean/ast_parser.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"789c40fc9b050fa1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}