{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/from-language-modeling-to-instruction","title":"From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning","arxiv_id":"2310.00492","date":"2023-09-30","proceeding":null,"authors":["Xuansheng Wu","Wenlin Yao","Jianshu Chen","Xiaoman Pan","Xiaoyang Wang","Ninghao Liu","Dong Yu"],"abstract":"Large Language Models (LLMs) have achieved remarkable success, where instruction tuning is the critical step in aligning LLMs with user intentions. In this work, we investigate how the instruction tuning adjusts pre-trained models with a focus on intrinsic changes. Specifically, we first develop several local and global explanation methods, including a gradient-based method for input-output attribution, and techniques for interpreting patterns and concepts in self-attention and feed-forward layers. The impact of instruction tuning is then studied by comparing the explanations derived from the pre-trained and instruction-tuned models. This approach provides an internal perspective of the model shifts on a human-comprehensible level. Our findings reveal three significant impacts of instruction tuning: 1) It empowers LLMs to recognize the instruction parts of user prompts, and promotes the response generation constantly conditioned on the instructions. 2) It encourages the self-attention heads to capture more word-word relationships about instruction verbs. 3) It encourages the feed-forward networks to rotate their pre-trained knowledge toward user-oriented tasks. These insights contribute to a more comprehensive understanding of instruction tuning and lay the groundwork for future work that aims at explaining and optimizing LLMs for various applications. Our code and data are publicly available at https://github.com/JacksonWuxs/Interpret_Instruction_Tuning_LLMs.","url_abs":"https://arxiv.org/abs/2310.00492v3","url_pdf":"https://arxiv.org/pdf/2310.00492v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"from-language-modeling-to-instruction","repo_url":"https://github.com/jacksonwuxs/interpret_instruction_tuning_llms","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[{"task_slug":"instruction-following","task_name":"Instruction Following"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"response-generation","task_name":"Response Generation"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.00492","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.00492"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jacksonwuxs/interpret_instruction_tuning_llms","reach":{"status":"ok","spdx":"GPL-3.0"}}],"summary":{"ran":5,"unverified":3},"by_repo_kind":{"official":{"samples":8,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"f805a9f405e9f2b0","entry":"KMPSearch","repo":"jacksonwuxs/interpret_instruction_tuning_llms","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/jacksonwuxs/interpret_instruction_tuning_llms/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"f805a9f405e9f2b0"}},{"code_sha256_prefix":"199ce04f205abafb","entry":"common_substrings","repo":"jacksonwuxs/interpret_instruction_tuning_llms","repo_kind":"official","path":"src/analysis_linear_interpret.py","file_url":"https://github.com/jacksonwuxs/interpret_instruction_tuning_llms/blob/HEAD/src/analysis_linear_interpret.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"199ce04f205abafb"}},{"code_sha256_prefix":"c29b6a708dc39670","entry":"decode","repo":"jacksonwuxs/interpret_instruction_tuning_llms","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/jacksonwuxs/interpret_instruction_tuning_llms/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"c29b6a708dc39670"}},{"code_sha256_prefix":"f964e2b69396a7c1","entry":"encode","repo":"jacksonwuxs/interpret_instruction_tuning_llms","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/jacksonwuxs/interpret_instruction_tuning_llms/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"f964e2b69396a7c1"}},{"code_sha256_prefix":"f543931abc30db2a","entry":"format_llama_weight","repo":"jacksonwuxs/interpret_instruction_tuning_llms","repo_kind":"official","path":"src/generator.py","file_url":"https://github.com/jacksonwuxs/interpret_instruction_tuning_llms/blob/HEAD/src/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"f543931abc30db2a"}},{"code_sha256_prefix":"932c3df6bd43ba46","entry":"ConceptInterpretor","repo":"jacksonwuxs/interpret_instruction_tuning_llms","repo_kind":"official","path":"src/run_chatgpt_annotate.py","file_url":"https://github.com/jacksonwuxs/interpret_instruction_tuning_llms/blob/HEAD/src/run_chatgpt_annotate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"932c3df6bd43ba46"}},{"code_sha256_prefix":"1f54fa0d9daacae5","entry":"NeuronInterpretor","repo":"jacksonwuxs/interpret_instruction_tuning_llms","repo_kind":"official","path":"src/run_chatgpt_annotate.py","file_url":"https://github.com/jacksonwuxs/interpret_instruction_tuning_llms/blob/HEAD/src/run_chatgpt_annotate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"1f54fa0d9daacae5"}},{"code_sha256_prefix":"8badb3485e88a7b1","entry":"TaskInterpretor","repo":"jacksonwuxs/interpret_instruction_tuning_llms","repo_kind":"official","path":"src/run_chatgpt_annotate.py","file_url":"https://github.com/jacksonwuxs/interpret_instruction_tuning_llms/blob/HEAD/src/run_chatgpt_annotate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"8badb3485e88a7b1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}