{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/improving-tree-structured-decoder-training","title":"Improving Tree-Structured Decoder Training for Code Generation via Mutual Learning","arxiv_id":"2105.14796","date":"2021-05-31","proceeding":null,"authors":["Binbin Xie","Jinsong Su","Yubin Ge","Xiang Li","Jianwei Cui","Junfeng Yao","Bin Wang"],"abstract":"Code generation aims to automatically generate a piece of code given an input natural language utterance. Currently, among dominant models, it is treated as a sequence-to-tree task, where a decoder outputs a sequence of actions corresponding to the pre-order traversal of an Abstract Syntax Tree. However, such a decoder only exploits the preorder traversal based preceding actions, which are insufficient to ensure correct action predictions. In this paper, we first throughly analyze the context modeling difference between neural code generation models with different traversals based decodings (preorder traversal vs breadth-first traversal), and then propose to introduce a mutual learning framework to jointly train these models. Under this framework, we continuously enhance both two models via mutual distillation, which involves synchronous executions of two one-to-one knowledge transfers at each training step. More specifically, we alternately choose one model as the student and the other as its teacher, and require the student to fit the training data and the action prediction distributions of its teacher. By doing so, both models can fully absorb the knowledge from each other and thus could be improved simultaneously. Experimental results and in-depth analysis on several benchmark datasets demonstrate the effectiveness of our approach. We release our code at https://github.com/DeepLearnXMU/CGML.","url_abs":"https://arxiv.org/abs/2105.14796v1","url_pdf":"https://arxiv.org/pdf/2105.14796v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"code-generation","task_name":"Code Generation"},{"task_slug":"decoder","task_name":"Decoder"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2105.14796","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2105.14796"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/DeepLearnXMU/CGML","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":7},"by_repo_kind":{"found_in_text":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fe716773de4c9a26","entry":"decode","repo":"DeepLearnXMU/CGML","repo_kind":"found_in_text","path":"evaluation.py","file_url":"https://github.com/DeepLearnXMU/CGML/blob/HEAD/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fe716773de4c9a26"}},{"code_sha256_prefix":"0ae89da9fe556223","entry":"dot_prod_attention","repo":"DeepLearnXMU/CGML","repo_kind":"found_in_text","path":"model/nn_utils.py","file_url":"https://github.com/DeepLearnXMU/CGML/blob/HEAD/model/nn_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0ae89da9fe556223"}},{"code_sha256_prefix":"5d62737e101413fe","entry":"evaluate","repo":"DeepLearnXMU/CGML","repo_kind":"found_in_text","path":"evaluation.py","file_url":"https://github.com/DeepLearnXMU/CGML/blob/HEAD/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5d62737e101413fe"}},{"code_sha256_prefix":"4d634e9ce2a4f738","entry":"input_transpose","repo":"DeepLearnXMU/CGML","repo_kind":"found_in_text","path":"model/nn_utils.py","file_url":"https://github.com/DeepLearnXMU/CGML/blob/HEAD/model/nn_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4d634e9ce2a4f738"}},{"code_sha256_prefix":"cebded354f300c3a","entry":"judgeParent","repo":"DeepLearnXMU/CGML","repo_kind":"found_in_text","path":"exp_distillation.py","file_url":"https://github.com/DeepLearnXMU/CGML/blob/HEAD/exp_distillation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cebded354f300c3a"}},{"code_sha256_prefix":"4f498904d7930e35","entry":"length_array_to_mask_tensor","repo":"DeepLearnXMU/CGML","repo_kind":"found_in_text","path":"model/nn_utils.py","file_url":"https://github.com/DeepLearnXMU/CGML/blob/HEAD/model/nn_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4f498904d7930e35"}},{"code_sha256_prefix":"c8ef224e9bbb07d6","entry":"search","repo":"DeepLearnXMU/CGML","repo_kind":"found_in_text","path":"exp_distillation.py","file_url":"https://github.com/DeepLearnXMU/CGML/blob/HEAD/exp_distillation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c8ef224e9bbb07d6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}