{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hard-prompts-made-interpretable-sparse","title":"Hard Prompts Made Interpretable: Sparse Entropy Regularization for Prompt Tuning with RL","arxiv_id":"2407.14733","date":"2024-07-20","proceeding":null,"authors":["Yunseon Choi","Sangmin Bae","Seonghyun Ban","Minchan Jeong","Chuheng Zhang","Lei Song","Li Zhao","Jiang Bian","Kee-Eung Kim"],"abstract":"With the advent of foundation models, prompt tuning has positioned itself as an important technique for directing model behaviors and eliciting desired responses. Prompt tuning regards selecting appropriate keywords included into the input, thereby adapting to the downstream task without adjusting or fine-tuning the model parameters. There is a wide range of work in prompt tuning, from approaches that directly harness the backpropagated gradient signals from the model, to those employing black-box optimization such as reinforcement learning (RL) methods. Our primary focus is on RLPrompt, which aims to find optimal prompt tokens leveraging soft Q-learning. While the results show promise, we have observed that the prompts frequently appear unnatural, which impedes their interpretability. We address this limitation by using sparse Tsallis entropy regularization, a principled approach to filtering out unlikely tokens from consideration. We extensively evaluate our approach across various tasks, including few-shot text classification, unsupervised text style transfer, and textual inversion from images. The results indicate a notable improvement over baselines, highlighting the efficacy of our approach in addressing the challenges of prompt tuning. Moreover, we show that the prompts discovered using our method are more natural and interpretable compared to those from other baselines.","url_abs":"https://arxiv.org/abs/2407.14733v1","url_pdf":"https://arxiv.org/pdf/2407.14733v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hard-prompts-made-interpretable-sparse","repo_url":"https://github.com/youseob/pin","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"few-shot-text-classification","task_name":"Few-Shot Text Classification"},{"task_slug":"q-learning","task_name":"Q-Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"style-transfer","task_name":"Style Transfer"},{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"text-style-transfoer","task_name":"Text Style Transfer"},{"task_slug":"unsupervised-text-style-transfer","task_name":"Unsupervised Text Style Transfer"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.14733","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.14733"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/youseob/pin","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"05aef45e369efe7f","entry":"sparse_max_operator","repo":"youseob/pin","repo_kind":"official","path":"rlprompt/losses/sparse_ql_losses.py","file_url":"https://github.com/youseob/pin/blob/HEAD/rlprompt/losses/sparse_ql_losses.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"05aef45e369efe7f"}},{"code_sha256_prefix":"78ecad8cb4340dea","entry":"sparse_ql_loss_1","repo":"youseob/pin","repo_kind":"official","path":"rlprompt/losses/sparse_ql_losses.py","file_url":"https://github.com/youseob/pin/blob/HEAD/rlprompt/losses/sparse_ql_losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"78ecad8cb4340dea"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}