{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/air-decoding-attribute-distribution","title":"Air-Decoding: Attribute Distribution Reconstruction for Decoding-Time Controllable Text Generation","arxiv_id":"2310.14892","date":"2023-10-23","proceeding":null,"authors":["Tianqi Zhong","Quan Wang","Jingxuan Han","Yongdong Zhang","Zhendong Mao"],"abstract":"Controllable text generation (CTG) aims to generate text with desired attributes, and decoding-time-based methods have shown promising performance on this task. However, in this paper, we identify the phenomenon of Attribute Collapse for the first time. It causes the fluency of generated text to rapidly decrease when the control strength exceeds a critical value, rendering the text completely unusable. This limitation hinders the effectiveness of decoding methods in achieving high levels of controllability. To address this problem, we propose a novel lightweight decoding framework named Air-Decoding. Its main idea is reconstructing the attribute distributions to balance the weights between attribute words and non-attribute words to generate more fluent text. Specifically, we train prefixes by prefix-tuning to obtain attribute distributions. Then we design a novel attribute distribution reconstruction method to balance the obtained distributions and use the reconstructed distributions to guide language models for generation, effectively avoiding the issue of Attribute Collapse. Experiments on multiple CTG tasks prove that our method achieves a new state-of-the-art control performance.","url_abs":"https://arxiv.org/abs/2310.14892v3","url_pdf":"https://arxiv.org/pdf/2310.14892v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"air-decoding-attribute-distribution","repo_url":"https://github.com/r1047/air-decoding","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.14892","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.14892"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/r1047/air-decoding","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":7,"unverified":1},"by_repo_kind":{"official":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"630cad41c958eab5","entry":"count_ngram","repo":"r1047/air-decoding","repo_kind":"official","path":"eval_dist.py","file_url":"https://github.com/r1047/air-decoding/blob/HEAD/eval_dist.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"630cad41c958eab5"}},{"code_sha256_prefix":"055f1d87c1438a30","entry":"eval_distinct","repo":"r1047/air-decoding","repo_kind":"official","path":"eval_dist.py","file_url":"https://github.com/r1047/air-decoding/blob/HEAD/eval_dist.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"055f1d87c1438a30"}},{"code_sha256_prefix":"81b160ae5a338b60","entry":"padding_fuse_fn","repo":"r1047/air-decoding","repo_kind":"official","path":"eval_sent_acc.py","file_url":"https://github.com/r1047/air-decoding/blob/HEAD/eval_sent_acc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"81b160ae5a338b60"}},{"code_sha256_prefix":"f6084b70931b4605","entry":"padding_fuse_fn","repo":"r1047/air-decoding","repo_kind":"official","path":"eval_topic_acc.py","file_url":"https://github.com/r1047/air-decoding/blob/HEAD/eval_topic_acc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f6084b70931b4605"}},{"code_sha256_prefix":"69d43732bc565135","entry":"padding_fuse_fn","repo":"r1047/air-decoding","repo_kind":"official","path":"train_PCLMs.py","file_url":"https://github.com/r1047/air-decoding/blob/HEAD/train_PCLMs.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"69d43732bc565135"}},{"code_sha256_prefix":"cd40a8cc4513a278","entry":"tokenized","repo":"r1047/air-decoding","repo_kind":"official","path":"eval_sent_acc.py","file_url":"https://github.com/r1047/air-decoding/blob/HEAD/eval_sent_acc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd40a8cc4513a278"}},{"code_sha256_prefix":"982e9a7c1be6b543","entry":"tokenized","repo":"r1047/air-decoding","repo_kind":"official","path":"eval_topic_acc.py","file_url":"https://github.com/r1047/air-decoding/blob/HEAD/eval_topic_acc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"982e9a7c1be6b543"}},{"code_sha256_prefix":"cbb2cea3d40f40de","entry":"cal_ppl","repo":"r1047/air-decoding","repo_kind":"official","path":"eval_perplexity.py","file_url":"https://github.com/r1047/air-decoding/blob/HEAD/eval_perplexity.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cbb2cea3d40f40de"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}