{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/constrained-decoding-for-secure-code","title":"Constrained Decoding for Secure Code Generation","arxiv_id":"2405.00218","date":"2024-04-30","proceeding":null,"authors":["Yanjun Fu","Ethan Baker","Yu Ding","Yizheng Chen"],"abstract":"Code Large Language Models (Code LLMs) have been increasingly used by developers to boost productivity, but they often generate vulnerable code. Thus, there is an urgent need to ensure that code generated by Code LLMs is correct and secure. Previous research has primarily focused on generating secure code, overlooking the fact that secure code also needs to be correct. This oversight can lead to a false sense of security. Currently, the community lacks a method to measure actual progress in this area, and we need solutions that address both security and correctness of code generation. This paper introduces a new benchmark, CodeGuard+, along with two new metrics, to measure Code LLMs' ability to generate both secure and correct code. Using our new evaluation methods, we show that the state-of-the-art defense technique, prefix tuning, may not be as strong as previously believed, since it generates secure code but sacrifices functional correctness. We also demonstrate that different decoding methods significantly affect the security of Code LLMs. Furthermore, we explore a new defense direction: constrained decoding for secure code generation. We propose new constrained decoding techniques to generate secure code. Our results reveal that constrained decoding is more effective than prefix tuning to improve the security of Code LLMs, without requiring a specialized training dataset. Moreover, our evaluations over eight state-of-the-art Code LLMs show that constrained decoding has strong performance to improve the security of Code LLMs, and our technique outperforms GPT-4.","url_abs":"https://arxiv.org/abs/2405.00218v3","url_pdf":"https://arxiv.org/pdf/2405.00218v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"constrained-decoding-for-secure-code","repo_url":"https://github.com/dynamite321/codeguardplus","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"constrained-decoding-for-secure-code","repo_url":"https://github.com/codeguardplus/codeguardplus","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"code-generation","task_name":"Code Generation"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-4","method_name":"GPT-4"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.00218","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.00218"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dynamite321/codeguardplus","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/codeguardplus/codeguardplus","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":8,"unverified":5},"by_repo_kind":{"official":{"samples":13,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4bd827a4fdf8d096","entry":"confidence_interval","repo":"dynamite321/codeguardplus","repo_kind":"official","path":"correctness_eval.py","file_url":"https://github.com/dynamite321/codeguardplus/blob/HEAD/correctness_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4bd827a4fdf8d096"}},{"code_sha256_prefix":"03e99c761c545617","entry":"duplicate_interleave","repo":"dynamite321/codeguardplus","repo_kind":"official","path":"inference/hf/modeling_codegen.py","file_url":"https://github.com/dynamite321/codeguardplus/blob/HEAD/inference/hf/modeling_codegen.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"03e99c761c545617"}},{"code_sha256_prefix":"7ea21b9da7ef5be6","entry":"fixed_pos_embedding","repo":"dynamite321/codeguardplus","repo_kind":"official","path":"inference/hf/modeling_codegen.py","file_url":"https://github.com/dynamite321/codeguardplus/blob/HEAD/inference/hf/modeling_codegen.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7ea21b9da7ef5be6"}},{"code_sha256_prefix":"5fe250b209299fb2","entry":"get_old_stats_json","repo":"dynamite321/codeguardplus","repo_kind":"official","path":"new_stats.py","file_url":"https://github.com/dynamite321/codeguardplus/blob/HEAD/new_stats.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5fe250b209299fb2"}},{"code_sha256_prefix":"718990946e149fe1","entry":"get_prompt","repo":"dynamite321/codeguardplus","repo_kind":"official","path":"correctness_eval.py","file_url":"https://github.com/dynamite321/codeguardplus/blob/HEAD/correctness_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"718990946e149fe1"}},{"code_sha256_prefix":"da54b6705bc5cee8","entry":"get_sonar_scans","repo":"dynamite321/codeguardplus","repo_kind":"official","path":"new_stats.py","file_url":"https://github.com/dynamite321/codeguardplus/blob/HEAD/new_stats.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"da54b6705bc5cee8"}},{"code_sha256_prefix":"aaf864304de22b06","entry":"pass_at_k","repo":"dynamite321/codeguardplus","repo_kind":"official","path":"correctness_eval.py","file_url":"https://github.com/dynamite321/codeguardplus/blob/HEAD/correctness_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aaf864304de22b06"}},{"code_sha256_prefix":"621a8a98538cd46d","entry":"rotate_every_two","repo":"dynamite321/codeguardplus","repo_kind":"official","path":"inference/hf/modeling_codegen.py","file_url":"https://github.com/dynamite321/codeguardplus/blob/HEAD/inference/hf/modeling_codegen.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"621a8a98538cd46d"}},{"code_sha256_prefix":"3ed1db97a93b601c","entry":"complete_scan","repo":"dynamite321/codeguardplus","repo_kind":"official","path":"sonar_scan.py","file_url":"https://github.com/dynamite321/codeguardplus/blob/HEAD/sonar_scan.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3ed1db97a93b601c"}},{"code_sha256_prefix":"e1b5a2d510fa54d0","entry":"config_from_pretrained","repo":"dynamite321/codeguardplus","repo_kind":"official","path":"inference/model.py","file_url":"https://github.com/dynamite321/codeguardplus/blob/HEAD/inference/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e1b5a2d510fa54d0"}},{"code_sha256_prefix":"957d15e0dd9a41e0","entry":"get_insecure_files","repo":"dynamite321/codeguardplus","repo_kind":"official","path":"sonar_scan.py","file_url":"https://github.com/dynamite321/codeguardplus/blob/HEAD/sonar_scan.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"957d15e0dd9a41e0"}},{"code_sha256_prefix":"8a519fe4005f6a16","entry":"new_project","repo":"dynamite321/codeguardplus","repo_kind":"official","path":"sonar_scan.py","file_url":"https://github.com/dynamite321/codeguardplus/blob/HEAD/sonar_scan.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8a519fe4005f6a16"}},{"code_sha256_prefix":"b147b5e666ea522a","entry":"try_parse","repo":"dynamite321/codeguardplus","repo_kind":"official","path":"codeql_eval.py","file_url":"https://github.com/dynamite321/codeguardplus/blob/HEAD/codeql_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b147b5e666ea522a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}