{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/consistent-autoformalization-for-constructing","title":"Consistent Autoformalization for Constructing Mathematical Libraries","arxiv_id":"2410.04194","date":"2024-10-05","proceeding":null,"authors":["Lan Zhang","Xin Quan","Andre Freitas"],"abstract":"Autoformalization is the task of automatically translating mathematical content written in natural language to a formal language expression. The growing language interpretation capabilities of Large Language Models (LLMs), including in formal languages, are lowering the barriers for autoformalization. However, LLMs alone are not capable of consistently and reliably delivering autoformalization, in particular as the complexity and specialization of the target domain grows. As the field evolves into the direction of systematically applying autoformalization towards large mathematical libraries, the need to improve syntactic, terminological and semantic control increases. This paper proposes the coordinated use of three mechanisms, most-similar retrieval augmented generation (MS-RAG), denoising steps, and auto-correction with syntax error feedback (Auto-SEF) to improve autoformalization quality. The empirical analysis, across different models, demonstrates that these mechanisms can deliver autoformalizaton results which are syntactically, terminologically and semantically more consistent. These mechanisms can be applied across different LLMs and have shown to deliver improve results across different model types.","url_abs":"https://arxiv.org/abs/2410.04194v1","url_pdf":"https://arxiv.org/pdf/2410.04194v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"consistent-autoformalization-for-constructing","repo_url":"https://github.com/lanzhang128/retrieval_augmented_autoformalization","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"rag","task_name":"RAG"},{"task_slug":"retrieval-augmented-generation","task_name":"Retrieval-augmented Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.04194","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.04194"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lanzhang128/retrieval_augmented_autoformalization","reach":{"status":"ok"}}],"summary":{"ran":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"369b7290587fc4a9","entry":"parse_error_file","repo":"lanzhang128/retrieval_augmented_autoformalization","repo_kind":"official","path":"isabelle/file_handler.py","file_url":"https://github.com/lanzhang128/retrieval_augmented_autoformalization/blob/HEAD/isabelle/file_handler.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"369b7290587fc4a9"}},{"code_sha256_prefix":"762336c532fafc50","entry":"parse_thy_file","repo":"lanzhang128/retrieval_augmented_autoformalization","repo_kind":"official","path":"IsarMathLib_extraction.py","file_url":"https://github.com/lanzhang128/retrieval_augmented_autoformalization/blob/HEAD/IsarMathLib_extraction.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"762336c532fafc50"}},{"code_sha256_prefix":"34659a4a896df0fa","entry":"postprocess_model_output","repo":"lanzhang128/retrieval_augmented_autoformalization","repo_kind":"official","path":"evaluation/utils.py","file_url":"https://github.com/lanzhang128/retrieval_augmented_autoformalization/blob/HEAD/evaluation/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"34659a4a896df0fa"}},{"code_sha256_prefix":"a302cd4737e235e7","entry":"preprocess_metric_input","repo":"lanzhang128/retrieval_augmented_autoformalization","repo_kind":"official","path":"evaluation/utils.py","file_url":"https://github.com/lanzhang128/retrieval_augmented_autoformalization/blob/HEAD/evaluation/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a302cd4737e235e7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}