{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/guiding-large-language-models-via-directional-1","title":"Guiding Large Language Models via Directional Stimulus Prompting","arxiv_id":"2302.11520","date":"2023-02-22","proceeding":"NeurIPS 2023 11","authors":["Zekun Li","Baolin Peng","Pengcheng He","Michel Galley","Jianfeng Gao","Xifeng Yan"],"abstract":"We introduce Directional Stimulus Prompting, a novel framework for guiding black-box large language models (LLMs) toward specific desired outputs. Instead of directly adjusting LLMs, our method employs a small tunable policy model (e.g., T5) to generate an auxiliary directional stimulus prompt for each input instance. These directional stimulus prompts act as nuanced, instance-specific hints and clues to guide LLMs in generating desired outcomes, such as including specific keywords in the generated summary. Our approach sidesteps the challenges of direct LLM tuning by optimizing the policy model to explore directional stimulus prompts that align LLMs with desired behaviors. The policy model can be optimized through 1) supervised fine-tuning using labeled data and 2) reinforcement learning from offline or online rewards based on the LLM's output. We assess our method across summarization, dialogue response generation, and chain-of-thought reasoning tasks. Our experiments demonstrate that the framework consistently improves LLMs' (e.g., ChatGPT, Codex, InstructGPT) performance on these supervised tasks using minimal labeled data. Notably, using just 80 dialogues on the MultiWOZ dataset, our approach enhances ChatGPT's performance by an impressive 41.4%, matching or surpassing some fully supervised start-of-the-art models. Additionally, the instance-specific chain-of-thought prompt generated by our approach improves InstructGPT's reasoning accuracy compared to human-crafted or automatically generated prompts. The code and data are publicly available at \\url{https://github.com/Leezekun/Directional-Stimulus-Prompting}.","url_abs":"https://arxiv.org/abs/2302.11520v4","url_pdf":"https://arxiv.org/pdf/2302.11520v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"guiding-large-language-models-via-directional-1","repo_url":"https://github.com/leezekun/directional-stimulus-prompting","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"response-generation","task_name":"Response Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2302.11520","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2302.11520"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/leezekun/directional-stimulus-prompting","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":6,"unverified":3},"by_repo_kind":{"official":{"samples":9,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"92e1118e4e5f774c","entry":"conjugate_gradient_solver","repo":"leezekun/directional-stimulus-prompting","repo_kind":"official","path":"rl4lms/algorithms/common/algo_utils.py","file_url":"https://github.com/leezekun/directional-stimulus-prompting/blob/HEAD/rl4lms/algorithms/common/algo_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"92e1118e4e5f774c"}},{"code_sha256_prefix":"91200094734b3131","entry":"find_products","repo":"leezekun/directional-stimulus-prompting","repo_kind":"official","path":"rl4lms/core_components/sweep.py","file_url":"https://github.com/leezekun/directional-stimulus-prompting/blob/HEAD/rl4lms/core_components/sweep.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"91200094734b3131"}},{"code_sha256_prefix":"206123801663c69f","entry":"flat_grad","repo":"leezekun/directional-stimulus-prompting","repo_kind":"official","path":"rl4lms/algorithms/common/algo_utils.py","file_url":"https://github.com/leezekun/directional-stimulus-prompting/blob/HEAD/rl4lms/algorithms/common/algo_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"206123801663c69f"}},{"code_sha256_prefix":"e38b5971551fdee2","entry":"get_dict_obj","repo":"leezekun/directional-stimulus-prompting","repo_kind":"official","path":"rl4lms/core_components/sweep.py","file_url":"https://github.com/leezekun/directional-stimulus-prompting/blob/HEAD/rl4lms/core_components/sweep.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e38b5971551fdee2"}},{"code_sha256_prefix":"821b0061bc694ddf","entry":"quantile_huber_loss","repo":"leezekun/directional-stimulus-prompting","repo_kind":"official","path":"rl4lms/algorithms/common/algo_utils.py","file_url":"https://github.com/leezekun/directional-stimulus-prompting/blob/HEAD/rl4lms/algorithms/common/algo_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"821b0061bc694ddf"}},{"code_sha256_prefix":"b41d462e09610b1c","entry":"to_expand","repo":"leezekun/directional-stimulus-prompting","repo_kind":"official","path":"rl4lms/core_components/sweep.py","file_url":"https://github.com/leezekun/directional-stimulus-prompting/blob/HEAD/rl4lms/core_components/sweep.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b41d462e09610b1c"}},{"code_sha256_prefix":"ea3e5b726884db6d","entry":"clean_text","repo":"leezekun/directional-stimulus-prompting","repo_kind":"official","path":"sft4lms/MultiWOZ/clean_dataset.py","file_url":"https://github.com/leezekun/directional-stimulus-prompting/blob/HEAD/sft4lms/MultiWOZ/clean_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ea3e5b726884db6d"}},{"code_sha256_prefix":"4c62f23ce8135bda","entry":"clean_time","repo":"leezekun/directional-stimulus-prompting","repo_kind":"official","path":"sft4lms/MultiWOZ/clean_dataset.py","file_url":"https://github.com/leezekun/directional-stimulus-prompting/blob/HEAD/sft4lms/MultiWOZ/clean_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4c62f23ce8135bda"}},{"code_sha256_prefix":"7f5a7dafc8e4429c","entry":"my_clean_text","repo":"leezekun/directional-stimulus-prompting","repo_kind":"official","path":"sft4lms/MultiWOZ/clean_dataset.py","file_url":"https://github.com/leezekun/directional-stimulus-prompting/blob/HEAD/sft4lms/MultiWOZ/clean_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7f5a7dafc8e4429c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}