{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/chat-vector-a-simple-approach-to-equip-llms","title":"Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languages","arxiv_id":"2310.04799","date":"2023-10-07","proceeding":null,"authors":["Shih-Cheng Huang","Pin-Zu Li","Yu-Chi Hsu","Kuang-Ming Chen","Yu Tung Lin","Shih-Kai Hsiao","Richard Tzong-Han Tsai","Hung-Yi Lee"],"abstract":"Recently, the development of open-source large language models (LLMs) has advanced rapidly. Nevertheless, due to data constraints, the capabilities of most open-source LLMs are primarily focused on English. To address this issue, we introduce the concept of $\\textit{chat vector}$ to equip pre-trained language models with instruction following and human value alignment via simple model arithmetic. The chat vector is derived by subtracting the weights of a pre-trained base model (e.g. LLaMA2) from those of its corresponding chat model (e.g. LLaMA2-chat). By simply adding the chat vector to a continual pre-trained model's weights, we can endow the model with chat capabilities in new languages without the need for further training. Our empirical studies demonstrate the superior efficacy of the chat vector from three different aspects: instruction following, toxicity mitigation, and multi-turn dialogue. Moreover, to showcase the adaptability of our approach, we extend our experiments to encompass various languages, base models, and chat vectors. The results underscore the chat vector's simplicity, effectiveness, and wide applicability, making it a compelling solution for efficiently enabling conversational capabilities in pre-trained language models. Our code is available at https://github.com/aqweteddy/ChatVector.","url_abs":"https://arxiv.org/abs/2310.04799v3","url_pdf":"https://arxiv.org/pdf/2310.04799v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"chat-vector-a-simple-approach-to-equip-llms","repo_url":"https://github.com/aqweteddy/ChatVector","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"chat-vector-a-simple-approach-to-equip-llms","repo_url":"https://github.com/augmxnt/deccp","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"instruction-following","task_name":"Instruction Following"}],"methods":[{"method_slug":"base","method_name":"BASE"},{"method_slug":"sft","method_name":"SFT"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2310.04799","atlas_url":"https://app.syntology.ai/?focus=2310.04799","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.04799"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aqweteddy/ChatVector","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/augmxnt/deccp","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":6,"unverified":1},"by_repo_kind":{"listed":{"samples":7,"ran":6,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"13db3ca133126100","entry":"add_chat_vector","repo":"aqweteddy/ChatVector","repo_kind":"listed","path":"add_chat_vector.py","file_url":"https://github.com/aqweteddy/ChatVector/blob/HEAD/add_chat_vector.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"13db3ca133126100"}},{"code_sha256_prefix":"984bb224cec763a0","entry":"direction_ablation_hook","repo":"augmxnt/deccp","repo_kind":"listed","path":"multilayer-inference.py","file_url":"https://github.com/augmxnt/deccp/blob/HEAD/multilayer-inference.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"984bb224cec763a0"}},{"code_sha256_prefix":"b3776edbed823b43","entry":"measure_fn","repo":"augmxnt/deccp","repo_kind":"listed","path":"abliterator.py","file_url":"https://github.com/augmxnt/deccp/blob/HEAD/abliterator.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b3776edbed823b43"}},{"code_sha256_prefix":"0a4275c9fbc4cb23","entry":"prepare_conversations","repo":"augmxnt/deccp","repo_kind":"listed","path":"tester/test-model.py","file_url":"https://github.com/augmxnt/deccp/blob/HEAD/tester/test-model.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0a4275c9fbc4cb23"}},{"code_sha256_prefix":"d53507b37cb81222","entry":"prepare_dataset","repo":"augmxnt/deccp","repo_kind":"listed","path":"abliterator.py","file_url":"https://github.com/augmxnt/deccp/blob/HEAD/abliterator.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d53507b37cb81222"}},{"code_sha256_prefix":"00d5313d11630f56","entry":"read_text_file","repo":"augmxnt/deccp","repo_kind":"listed","path":"upload-dataset.py","file_url":"https://github.com/augmxnt/deccp/blob/HEAD/upload-dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"00d5313d11630f56"}},{"code_sha256_prefix":"00f1c03855cad957","entry":"get_pipeline","repo":"aqweteddy/ChatVector","repo_kind":"listed","path":"chat.py","file_url":"https://github.com/aqweteddy/ChatVector/blob/HEAD/chat.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"00f1c03855cad957"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}