{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/codechain-towards-modular-code-generation","title":"CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules","arxiv_id":"2310.08992","date":"2023-10-13","proceeding":null,"authors":["Hung Le","Hailin Chen","Amrita Saha","Akash Gokul","Doyen Sahoo","Shafiq Joty"],"abstract":"Large Language Models (LLMs) have already become quite proficient at solving simpler programming tasks like those in HumanEval or MBPP benchmarks. However, solving more complex and competitive programming tasks is still quite challenging for these models - possibly due to their tendency to generate solutions as monolithic code blocks instead of decomposing them into logical sub-tasks and sub-modules. On the other hand, experienced programmers instinctively write modularized code with abstraction for solving complex tasks, often reusing previously developed modules. To address this gap, we propose CodeChain, a novel framework for inference that elicits modularized code generation through a chain of self-revisions, each being guided by some representative sub-modules generated in previous iterations. Concretely, CodeChain first instructs the LLM to generate modularized codes through chain-of-thought prompting. Then it applies a chain of self-revisions by iterating the two steps: 1) extracting and clustering the generated sub-modules and selecting the cluster representatives as the more generic and re-usable implementations, and 2) augmenting the original chain-of-thought prompt with these selected module-implementations and instructing the LLM to re-generate new modularized solutions. We find that by naturally encouraging the LLM to reuse the previously developed and verified sub-modules, CodeChain can significantly boost both modularity as well as correctness of the generated solutions, achieving relative pass@1 improvements of 35% on APPS and 76% on CodeContests. It is shown to be effective on both OpenAI LLMs as well as open-sourced LLMs like WizardCoder. We also conduct comprehensive ablation studies with different methods of prompting, number of clusters, model sizes, program qualities, etc., to provide useful insights that underpin CodeChain's success.","url_abs":"https://arxiv.org/abs/2310.08992v3","url_pdf":"https://arxiv.org/pdf/2310.08992v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"codechain-towards-modular-code-generation","repo_url":"https://github.com/SalesforceAIResearch/CodeChain","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"code-generation","task_name":"Code Generation"},{"task_slug":"humaneval","task_name":"HumanEval"},{"task_slug":null,"task_name":"mbpp"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/code-generation-on-apps","task":"Code Generation","dataset":"APPS","model":"CodeChain+WizardCoder-15b","rank_in_archive_order":7,"of":18,"metrics":{"Competition Pass@1":"2.5%","Competition Pass@any":"14.5%","Interview Pass@1":"6.4%","Interview Pass@any":"25.4%","Introductory Pass@1":"29.3%","Introductory Pass@any":"60.9%"},"uses_additional_data":false},{"leaderboard":"/sota/code-generation-on-apps","task":"Code Generation","dataset":"APPS","model":"WizardCoder-15b","rank_in_archive_order":8,"of":18,"metrics":{"Competition Pass@1":"3.75","Interview Pass@1":"7.49","Introductory Pass@1":"26.29"},"uses_additional_data":false},{"leaderboard":"/sota/code-generation-on-codecontests","task":"Code Generation","dataset":"CodeContests","model":"CodeChain + WizardCoder-15B","rank_in_archive_order":7,"of":8,"metrics":{"Test Set pass@1":"2.35","Test Set pass@5":"3.29","Val Set pass@1":"2.48","Val Set pass@5":"3.30"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2310.08992","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.08992"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SalesforceAIResearch/CodeChain","reach":null}],"summary":{"ran":1,"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f34f883bb6fb5e60","entry":"BaseEncoder","repo":"SalesforceAIResearch/CodeChain","repo_kind":"official","path":"src/embedding/encoder.py","file_url":"https://github.com/SalesforceAIResearch/CodeChain/blob/HEAD/src/embedding/encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f34f883bb6fb5e60"}},{"code_sha256_prefix":"55cffac4dad16773","entry":"prepare_tokenizer","repo":"SalesforceAIResearch/CodeChain","repo_kind":"official","path":"src/embedding/encoder.py","file_url":"https://github.com/SalesforceAIResearch/CodeChain/blob/HEAD/src/embedding/encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"55cffac4dad16773"}},{"code_sha256_prefix":"7396327c9cd9f836","entry":"truncate_sentences","repo":"SalesforceAIResearch/CodeChain","repo_kind":"official","path":"src/embedding/encoder.py","file_url":"https://github.com/SalesforceAIResearch/CodeChain/blob/HEAD/src/embedding/encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7396327c9cd9f836"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}