{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/transcoder-towards-unified-transferable-code","title":"TransCoder: Towards Unified Transferable Code Representation Learning Inspired by Human Skills","arxiv_id":"2306.07285","date":"2023-05-23","proceeding":null,"authors":["Qiushi Sun","Nuo Chen","Jianing Wang","Xiang Li","Ming Gao"],"abstract":"Code pre-trained models (CodePTMs) have recently demonstrated a solid capacity to process various software intelligence tasks, e.g., code clone detection, code translation, and code summarization. The current mainstream method that deploys these models to downstream tasks is to fine-tune them on individual tasks, which is generally costly and needs sufficient data for large models. To tackle the issue, in this paper, we present TransCoder, a unified Transferable fine-tuning strategy for Code representation learning. Inspired by human inherent skills of knowledge generalization, TransCoder drives the model to learn better code-related meta-knowledge like human programmers. Specifically, we employ a tunable prefix encoder as the meta-learner to capture cross-task and cross-language transferable knowledge, respectively. Besides, tasks with minor training sample sizes and languages with small corpus can be remarkably benefited from our approach. Extensive experiments conducted on benchmark datasets clearly demonstrate that our method can lead to superior performance on various code-related tasks and encourage mutual reinforcement. We also show that TransCoder is applicable in low-resource scenarios. Our codes are available at https://github.com/QiushiSun/TransCoder.","url_abs":"https://arxiv.org/abs/2306.07285v2","url_pdf":"https://arxiv.org/pdf/2306.07285v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"transcoder-towards-unified-transferable-code","repo_url":"https://github.com/qiushisun/transcoder","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"clone-detection","task_name":"Clone Detection"},{"task_slug":"code-summarization-1","task_name":"Code Summarization"},{"task_slug":"code-translation","task_name":"Code Translation"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.07285","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.07285"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qiushisun/transcoder","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":9},"by_repo_kind":{"official":{"samples":10,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8cfe38019e9d6a49","entry":"get_model_size","repo":"qiushisun/transcoder","repo_kind":"official","path":"models_2.py","file_url":"https://github.com/qiushisun/transcoder/blob/HEAD/models_2.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8cfe38019e9d6a49"}},{"code_sha256_prefix":"b853ebf5b01e9047","entry":"add_args","repo":"qiushisun/transcoder","repo_kind":"official","path":"configs.py","file_url":"https://github.com/qiushisun/transcoder/blob/HEAD/configs.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b853ebf5b01e9047"}},{"code_sha256_prefix":"d313ee74f207a251","entry":"add_args","repo":"qiushisun/transcoder","repo_kind":"official","path":"configs_fewshot.py","file_url":"https://github.com/qiushisun/transcoder/blob/HEAD/configs_fewshot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d313ee74f207a251"}},{"code_sha256_prefix":"d9d8ed639ae7e5d0","entry":"bulid_or_load_gen_model","repo":"qiushisun/transcoder","repo_kind":"official","path":"models_2.py","file_url":"https://github.com/qiushisun/transcoder/blob/HEAD/models_2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9d8ed639ae7e5d0"}},{"code_sha256_prefix":"26c1a357cded8c11","entry":"eval_ppl_epoch","repo":"qiushisun/transcoder","repo_kind":"official","path":"learner.py","file_url":"https://github.com/qiushisun/transcoder/blob/HEAD/learner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26c1a357cded8c11"}},{"code_sha256_prefix":"61be214eec2c0bb2","entry":"evaluate_cls","repo":"qiushisun/transcoder","repo_kind":"official","path":"learner.py","file_url":"https://github.com/qiushisun/transcoder/blob/HEAD/learner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61be214eec2c0bb2"}},{"code_sha256_prefix":"a719263e88013c50","entry":"get_model_size","repo":"qiushisun/transcoder","repo_kind":"official","path":"models.py","file_url":"https://github.com/qiushisun/transcoder/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a719263e88013c50"}},{"code_sha256_prefix":"3824baa1aca5bb25","entry":"get_sample_size","repo":"qiushisun/transcoder","repo_kind":"official","path":"learner.py","file_url":"https://github.com/qiushisun/transcoder/blob/HEAD/learner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3824baa1aca5bb25"}},{"code_sha256_prefix":"bfbc7746415f3426","entry":"load_tf_weights_in_t5","repo":"qiushisun/transcoder","repo_kind":"official","path":"models_list/T5ForConditionalGeneration_Prefix_2.py","file_url":"https://github.com/qiushisun/transcoder/blob/HEAD/models_list/T5ForConditionalGeneration_Prefix_2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bfbc7746415f3426"}},{"code_sha256_prefix":"417e73bc0c3843bb","entry":"shift_tokens_right","repo":"qiushisun/transcoder","repo_kind":"official","path":"models_list/PLBartForConditionalGeneration_Prefix.py","file_url":"https://github.com/qiushisun/transcoder/blob/HEAD/models_list/PLBartForConditionalGeneration_Prefix.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"417e73bc0c3843bb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}