{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/instruction-tuning-for-secure-code-generation","title":"Instruction Tuning for Secure Code Generation","arxiv_id":"2402.09497","date":"2024-02-14","proceeding":null,"authors":["Jingxuan He","Mark Vero","Gabriela Krasnopolska","Martin Vechev"],"abstract":"Modern language models (LMs) have gained widespread acceptance in everyday and professional contexts, particularly in programming. An essential procedure enabling this adoption is instruction tuning, which substantially enhances LMs' practical utility by training them to follow user instructions and human preferences. However, existing instruction tuning schemes overlook a crucial aspect: the security of generated code. As a result, even the state-of-the-art instruction-tuned LMs frequently produce unsafe code, posing significant security risks. In this work, we introduce SafeCoder to address this gap. SafeCoder performs security-centric fine-tuning using a diverse and high-quality dataset that we collected using an automated pipeline. We integrate the security fine-tuning with standard instruction tuning, to facilitate a joint optimization of both security and utility. Despite its simplicity, we show that SafeCoder is effective across a variety of popular LMs and datasets. It is able to drastically improve security (by about 30%), while preserving utility.","url_abs":"https://arxiv.org/abs/2402.09497v2","url_pdf":"https://arxiv.org/pdf/2402.09497v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"instruction-tuning-for-secure-code-generation","repo_url":"https://github.com/eth-sri/safecoder","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"code-generation","task_name":"Code Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.09497","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.09497"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/eth-sri/safecoder","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":9,"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":11,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"247d5669b5fd3163","entry":"eval_script_python","repo":"eth-sri/safecoder","repo_kind":"official","path":"safecoder/human_eval/containerized_eval.py","file_url":"https://github.com/eth-sri/safecoder/blob/HEAD/safecoder/human_eval/containerized_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"247d5669b5fd3163"}},{"code_sha256_prefix":"b1eca2a5e7fedc04","entry":"eval_string_script","repo":"eth-sri/safecoder","repo_kind":"official","path":"safecoder/human_eval/containerized_eval.py","file_url":"https://github.com/eth-sri/safecoder/blob/HEAD/safecoder/human_eval/containerized_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1eca2a5e7fedc04"}},{"code_sha256_prefix":"22127156c00606cd","entry":"get_cp_args","repo":"eth-sri/safecoder","repo_kind":"official","path":"safecoder/utils.py","file_url":"https://github.com/eth-sri/safecoder/blob/HEAD/safecoder/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"22127156c00606cd"}},{"code_sha256_prefix":"ad7d9a005c32f9ae","entry":"get_logits_from_lm","repo":"eth-sri/safecoder","repo_kind":"official","path":"safecoder/trainer.py","file_url":"https://github.com/eth-sri/safecoder/blob/HEAD/safecoder/trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad7d9a005c32f9ae"}},{"code_sha256_prefix":"3a7c7ec16769928a","entry":"pass_at_k","repo":"eth-sri/safecoder","repo_kind":"official","path":"safecoder/metric.py","file_url":"https://github.com/eth-sri/safecoder/blob/HEAD/safecoder/metric.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3a7c7ec16769928a"}},{"code_sha256_prefix":"7ba8f0a6db856c7d","entry":"token_weighted_loss","repo":"eth-sri/safecoder","repo_kind":"official","path":"safecoder/trainer.py","file_url":"https://github.com/eth-sri/safecoder/blob/HEAD/safecoder/trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7ba8f0a6db856c7d"}},{"code_sha256_prefix":"780f982b0a1c9b3a","entry":"truncate_after","repo":"eth-sri/safecoder","repo_kind":"official","path":"safecoder/evaler.py","file_url":"https://github.com/eth-sri/safecoder/blob/HEAD/safecoder/evaler.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"780f982b0a1c9b3a"}},{"code_sha256_prefix":"f58ac813b10f0912","entry":"truncate_after_last","repo":"eth-sri/safecoder","repo_kind":"official","path":"safecoder/evaler.py","file_url":"https://github.com/eth-sri/safecoder/blob/HEAD/safecoder/evaler.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f58ac813b10f0912"}},{"code_sha256_prefix":"99c28062b34f83e2","entry":"truncate_before","repo":"eth-sri/safecoder","repo_kind":"official","path":"safecoder/evaler.py","file_url":"https://github.com/eth-sri/safecoder/blob/HEAD/safecoder/evaler.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"99c28062b34f83e2"}},{"code_sha256_prefix":"dcbf19c0a49959fb","entry":"try_parse","repo":"eth-sri/safecoder","repo_kind":"official","path":"safecoder/utils.py","file_url":"https://github.com/eth-sri/safecoder/blob/HEAD/safecoder/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dcbf19c0a49959fb"}},{"code_sha256_prefix":"a477d8eb91a8ac1b","entry":"load_model","repo":"eth-sri/safecoder","repo_kind":"official","path":"safecoder/utils.py","file_url":"https://github.com/eth-sri/safecoder/blob/HEAD/safecoder/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a477d8eb91a8ac1b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}