{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cold-decoding-energy-based-constrained-text","title":"COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics","arxiv_id":"2202.11705","date":"2022-02-23","proceeding":null,"authors":["Lianhui Qin","Sean Welleck","Daniel Khashabi","Yejin Choi"],"abstract":"Many applications of text generation require incorporating different constraints to control the semantics or style of generated text. These constraints can be hard (e.g., ensuring certain keywords are included in the output) and soft (e.g., contextualizing the output with the left- or right-hand context). In this paper, we present Energy-based Constrained Decoding with Langevin Dynamics (COLD), a decoding framework which unifies constrained generation as specifying constraints through an energy function, then performing efficient differentiable reasoning over the constraints through gradient-based sampling. COLD decoding is a flexible framework that can be applied directly to off-the-shelf left-to-right language models without the need for any task-specific fine-tuning, as demonstrated through three challenging text generation applications: lexically-constrained generation, abductive reasoning, and counterfactual reasoning. Our experiments on these constrained generation tasks point to the effectiveness of our approach, both in terms of automatic and human evaluation.","url_abs":"https://arxiv.org/abs/2202.11705v3","url_pdf":"https://arxiv.org/pdf/2202.11705v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cold-decoding-energy-based-constrained-text","repo_url":"https://github.com/qkaren/cold_decoding","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"cold-decoding-energy-based-constrained-text","repo_url":"https://github.com/yu-fangxu/cold-attack","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"counterfactual-reasoning","task_name":"Counterfactual Reasoning"},{"task_slug":"text-generation","task_name":"Text Generation"},{"task_slug":null,"task_name":"counterfactual"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2202.11705","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2202.11705"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/qkaren/COLD_decoding","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qkaren/cold_decoding","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yu-fangxu/cold-attack","reach":{"status":"ok"}}],"summary":{"ran":1,"unverified":5},"by_repo_kind":{"official":{"samples":6,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d4931205d88652cd","entry":"batch_log_bleulosscnn_ae","repo":"qkaren/COLD_decoding","repo_kind":"official","path":"bleuloss.py","file_url":"https://github.com/qkaren/COLD_decoding/blob/HEAD/bleuloss.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d4931205d88652cd"}},{"code_sha256_prefix":"b2222118d299d485","entry":"embed_inputs","repo":"qkaren/COLD_decoding","repo_kind":"official","path":"util.py","file_url":"https://github.com/qkaren/COLD_decoding/blob/HEAD/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b2222118d299d485"}},{"code_sha256_prefix":"4fce6cee9a3fe85d","entry":"gelu","repo":"qkaren/COLD_decoding","repo_kind":"official","path":"GPT2ForwardBackward/modeling_opengpt2.py","file_url":"https://github.com/qkaren/COLD_decoding/blob/HEAD/GPT2ForwardBackward/modeling_opengpt2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4fce6cee9a3fe85d"}},{"code_sha256_prefix":"00a33466c69c5705","entry":"load_tf_weights_in_gpt2","repo":"qkaren/COLD_decoding","repo_kind":"official","path":"GPT2ForwardBackward/modeling_opengpt2.py","file_url":"https://github.com/qkaren/COLD_decoding/blob/HEAD/GPT2ForwardBackward/modeling_opengpt2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"00a33466c69c5705"}},{"code_sha256_prefix":"00c3cfb8a63fc383","entry":"top_k_filter","repo":"qkaren/COLD_decoding","repo_kind":"official","path":"util.py","file_url":"https://github.com/qkaren/COLD_decoding/blob/HEAD/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"00c3cfb8a63fc383"}},{"code_sha256_prefix":"a2c41a11d9bc159a","entry":"top_k_filter_3d","repo":"qkaren/COLD_decoding","repo_kind":"official","path":"util.py","file_url":"https://github.com/qkaren/COLD_decoding/blob/HEAD/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a2c41a11d9bc159a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}