{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/case2code-learning-inductive-reasoning-with","title":"Case2Code: Learning Inductive Reasoning with Synthetic Data","arxiv_id":"2407.12504","date":"2024-07-17","proceeding":null,"authors":["Yunfan Shao","Linyang Li","Yichuan Ma","Peiji Li","Demin Song","Qinyuan Cheng","ShiMin Li","Xiaonan Li","Pengyu Wang","Qipeng Guo","Hang Yan","Xipeng Qiu","Xuanjing Huang","Dahua Lin"],"abstract":"Complex reasoning is an impressive ability shown by large language models (LLMs). Most LLMs are skilled in deductive reasoning, such as chain-of-thought prompting or iterative tool-using to solve challenging tasks step-by-step. In this paper, we hope to focus on evaluating and teaching LLMs to conduct inductive reasoning, that is, LLMs are supposed to infer underlying rules by observing examples or sequential transformations. However, collecting large-scale and diverse human-generated inductive data is challenging. We focus on data synthesis in the code domain and propose a \\textbf{Case2Code} task by exploiting the expressiveness and correctness of programs. Specifically, we collect a diverse set of executable programs, synthesize input-output transformations for each program, and force LLMs to infer the underlying code implementations based on the synthetic I/O cases. We first evaluate representative LLMs on the synthesized Case2Code task and demonstrate that the Case-to-code induction is challenging for LLMs. Then, we synthesize large-scale Case2Code training samples to train LLMs to perform inductive reasoning. Experimental results show that such induction training benefits not only in distribution Case2Code performance but also enhances various coding abilities of trained LLMs, demonstrating the great potential of learning inductive reasoning via synthetic data.","url_abs":"https://arxiv.org/abs/2407.12504v1","url_pdf":"https://arxiv.org/pdf/2407.12504v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"case2code-learning-inductive-reasoning-with","repo_url":"https://github.com/choosewhatulike/case2code","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[{"method_slug":"focus","method_name":"Focus"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.12504","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.12504"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/choosewhatulike/case2code","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":9,"unverified":4},"by_repo_kind":{"official":{"samples":13,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"cbe1465055781766","entry":"execute_code_wrapped","repo":"choosewhatulike/case2code","repo_kind":"official","path":"exec_utils.py","file_url":"https://github.com/choosewhatulike/case2code/blob/HEAD/exec_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cbe1465055781766"}},{"code_sha256_prefix":"f77c3ab6fcec0050","entry":"find_next_matching_parentheses","repo":"choosewhatulike/case2code","repo_kind":"official","path":"prompt_code_data.py","file_url":"https://github.com/choosewhatulike/case2code/blob/HEAD/prompt_code_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f77c3ab6fcec0050"}},{"code_sha256_prefix":"f715b02aee57f4cc","entry":"get_functions","repo":"choosewhatulike/case2code","repo_kind":"official","path":"ast_utils.py","file_url":"https://github.com/choosewhatulike/case2code/blob/HEAD/ast_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f715b02aee57f4cc"}},{"code_sha256_prefix":"4a9ac795913154cf","entry":"hinted_tuple_hook","repo":"choosewhatulike/case2code","repo_kind":"official","path":"exec_utils.py","file_url":"https://github.com/choosewhatulike/case2code/blob/HEAD/exec_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4a9ac795913154cf"}},{"code_sha256_prefix":"ad5f98bba0c21929","entry":"read_json","repo":"choosewhatulike/case2code","repo_kind":"official","path":"api_call_util.py","file_url":"https://github.com/choosewhatulike/case2code/blob/HEAD/api_call_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ad5f98bba0c21929"}},{"code_sha256_prefix":"65c57366104cbd60","entry":"read_jsonl","repo":"choosewhatulike/case2code","repo_kind":"official","path":"api_call_util.py","file_url":"https://github.com/choosewhatulike/case2code/blob/HEAD/api_call_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"65c57366104cbd60"}},{"code_sha256_prefix":"91900a7213500feb","entry":"run_code_carefully","repo":"choosewhatulike/case2code","repo_kind":"official","path":"exec_utils.py","file_url":"https://github.com/choosewhatulike/case2code/blob/HEAD/exec_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"91900a7213500feb"}},{"code_sha256_prefix":"9d97a25d8bf47524","entry":"source_to_ast","repo":"choosewhatulike/case2code","repo_kind":"official","path":"ast_utils.py","file_url":"https://github.com/choosewhatulike/case2code/blob/HEAD/ast_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9d97a25d8bf47524"}},{"code_sha256_prefix":"1d3bfb6057c12f45","entry":"unwrap_code","repo":"choosewhatulike/case2code","repo_kind":"official","path":"data_process.py","file_url":"https://github.com/choosewhatulike/case2code/blob/HEAD/data_process.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1d3bfb6057c12f45"}},{"code_sha256_prefix":"77994d7607b4f856","entry":"get_data_processor","repo":"choosewhatulike/case2code","repo_kind":"official","path":"data_process.py","file_url":"https://github.com/choosewhatulike/case2code/blob/HEAD/data_process.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"77994d7607b4f856"}},{"code_sha256_prefix":"7e8e2b21d03cf0aa","entry":"process_code","repo":"choosewhatulike/case2code","repo_kind":"official","path":"mp_exec_code.py","file_url":"https://github.com/choosewhatulike/case2code/blob/HEAD/mp_exec_code.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7e8e2b21d03cf0aa"}},{"code_sha256_prefix":"c8f61b44d28f3ca2","entry":"read_gen_data","repo":"choosewhatulike/case2code","repo_kind":"official","path":"api_call_util.py","file_url":"https://github.com/choosewhatulike/case2code/blob/HEAD/api_call_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c8f61b44d28f3ca2"}},{"code_sha256_prefix":"b1f9298be41ff962","entry":"run_worker","repo":"choosewhatulike/case2code","repo_kind":"official","path":"mp_test_case2code_with_exe.py","file_url":"https://github.com/choosewhatulike/case2code/blob/HEAD/mp_test_case2code_with_exe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b1f9298be41ff962"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}