{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-code-preference-via-synthetic","title":"Learning Code Preference via Synthetic Evolution","arxiv_id":"2410.03837","date":"2024-10-04","proceeding":null,"authors":["Jiawei Liu","Thanh Nguyen","Mingyue Shang","Hantian Ding","Xiaopeng Li","Yu Yu","Varun Kumar","Zijian Wang"],"abstract":"Large Language Models (LLMs) have recently demonstrated remarkable coding capabilities. However, assessing code generation based on well-formed properties and aligning it with developer preferences remains challenging. In this paper, we explore two key questions under the new challenge of code preference learning: (i) How do we train models to predict meaningful preferences for code? and (ii) How do human and LLM preferences align with verifiable code properties and developer code tastes? To this end, we propose CodeFavor, a framework for training pairwise code preference models from synthetic evolution data, including code commits and code critiques. To evaluate code preferences, we introduce CodePrefBench, a benchmark comprising 1364 rigorously curated code preference tasks to cover three verifiable properties-correctness, efficiency, and security-along with human preference. Our evaluation shows that CodeFavor holistically improves the accuracy of model-based code preferences by up to 28.8%. Meanwhile, CodeFavor models can match the performance of models with 6-9x more parameters while being 34x more cost-effective. We also rigorously validate the design choices in CodeFavor via a comprehensive set of controlled experiments. Furthermore, we discover the prohibitive costs and limitations of human-based code preference: despite spending 23.4 person-minutes on each task, 15.1-40.3% of tasks remain unsolved. Compared to model-based preference, human preference tends to be more accurate under the objective of code correctness, while being sub-optimal for non-functional objectives.","url_abs":"https://arxiv.org/abs/2410.03837v2","url_pdf":"https://arxiv.org/pdf/2410.03837v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-code-preference-via-synthetic","repo_url":"https://github.com/amazon-science/llm-code-preference","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"code-generation","task_name":"Code Generation"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.03837","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.03837"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/amazon-science/llm-code-preference","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":8,"unverified":1},"by_repo_kind":{"official":{"samples":9,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"3a40321ab74c0bc2","entry":"datagen_inference","repo":"amazon-science/llm-code-preference","repo_kind":"official","path":"codefavor/prompt/critic_evol.py","file_url":"https://github.com/amazon-science/llm-code-preference/blob/HEAD/codefavor/prompt/critic_evol.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3a40321ab74c0bc2"}},{"code_sha256_prefix":"898b72c5ee01636a","entry":"fewshot_template_improve","repo":"amazon-science/llm-code-preference","repo_kind":"official","path":"codefavor/prompt/critic_evol.py","file_url":"https://github.com/amazon-science/llm-code-preference/blob/HEAD/codefavor/prompt/critic_evol.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"898b72c5ee01636a"}},{"code_sha256_prefix":"fc13e41d350f62cb","entry":"fewshot_template_no_improve","repo":"amazon-science/llm-code-preference","repo_kind":"official","path":"codefavor/prompt/critic_evol.py","file_url":"https://github.com/amazon-science/llm-code-preference/blob/HEAD/codefavor/prompt/critic_evol.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"fc13e41d350f62cb"}},{"code_sha256_prefix":"bfb3f9ec4b555806","entry":"hacky_assistant_stop_seq","repo":"amazon-science/llm-code-preference","repo_kind":"official","path":"codefavor/provider/base.py","file_url":"https://github.com/amazon-science/llm-code-preference/blob/HEAD/codefavor/provider/base.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"bfb3f9ec4b555806"}},{"code_sha256_prefix":"74e1c40ca345d7ce","entry":"pairwise_classification_template","repo":"amazon-science/llm-code-preference","repo_kind":"official","path":"codefavor/template.py","file_url":"https://github.com/amazon-science/llm-code-preference/blob/HEAD/codefavor/template.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"74e1c40ca345d7ce"}},{"code_sha256_prefix":"da043ad281c84014","entry":"pairwise_cot_template","repo":"amazon-science/llm-code-preference","repo_kind":"official","path":"codefavor/template.py","file_url":"https://github.com/amazon-science/llm-code-preference/blob/HEAD/codefavor/template.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"da043ad281c84014"}},{"code_sha256_prefix":"58badd651ef12b99","entry":"response_parser","repo":"amazon-science/llm-code-preference","repo_kind":"official","path":"codefavor/evaluate.py","file_url":"https://github.com/amazon-science/llm-code-preference/blob/HEAD/codefavor/evaluate.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"58badd651ef12b99"}},{"code_sha256_prefix":"1238c597ba115abe","entry":"task_md5","repo":"amazon-science/llm-code-preference","repo_kind":"official","path":"codefavor/evaluate.py","file_url":"https://github.com/amazon-science/llm-code-preference/blob/HEAD/codefavor/evaluate.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"1238c597ba115abe"}},{"code_sha256_prefix":"081407e2793ad7f0","entry":"merge_models","repo":"amazon-science/llm-code-preference","repo_kind":"official","path":"codefavor/merge_model.py","file_url":"https://github.com/amazon-science/llm-code-preference/blob/HEAD/codefavor/merge_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"081407e2793ad7f0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}