{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/improving-translation-faithfulness-of-large","title":"Improving Translation Faithfulness of Large Language Models via Augmenting Instructions","arxiv_id":"2308.12674","date":"2023-08-24","proceeding":null,"authors":["Yijie Chen","Yijin Liu","Fandong Meng","Yufeng Chen","Jinan Xu","Jie zhou"],"abstract":"Large Language Models (LLMs) present strong general capabilities, and a current compelling challenge is stimulating their specialized capabilities, such as machine translation, through low-cost instruction tuning. The standard instruction-following data is sequentially organized as the concatenation of an instruction, an input, and a response. As the attention mechanism of LLMs has limitations on local focus, LLMs tend to focus more on the words or sentences nearby at each position. This leads to a high risk of instruction forgetting during decoding. To alleviate the above issues, We propose SWIE (Segment-Weighted Instruction Embedding) and an instruction-following dataset OVERMISS. SWIE improves the model instruction understanding by adding a global instruction representation on the following input and response representations. OVERMISS improves model faithfulness by comparing over-translation and miss-translation results with the correct translation. We apply our methods to two main-stream open-source LLMs, BLOOM and LLaMA. The experimental results demonstrate significant improvements in translation performance with SWIE based on BLOOMZ-3b, particularly in zero-shot and long text translations due to reduced instruction forgetting risk. Additionally, OVERMISS outperforms the baseline in translation performance (e.g. an increase in BLEU scores from 0.69 to 3.12 and an average improvement of 0.48 percentage comet scores for LLaMA-7b) with further enhancements seen in models combining OVERMISS and SWIE (e.g. the BLUE scores increase up to 0.56 from English to German across three different backbones), and both exhibit improvements in the faithfulness metric based on word alignment.","url_abs":"https://arxiv.org/abs/2308.12674v1","url_pdf":"https://arxiv.org/pdf/2308.12674v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"improving-translation-faithfulness-of-large","repo_url":"https://github.com/pppa2019/swie_overmiss_llm4mt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"instruction-following","task_name":"Instruction Following"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"translation","task_name":"Translation"},{"task_slug":"word-alignment","task_name":"Word Alignment"}],"methods":[{"method_slug":"bloom","method_name":"BLOOM"},{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2308.12674","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.12674"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pppa2019/swie_overmiss_llm4mt","reach":{"status":"ok"}}],"summary":{"ran_fixture":1,"ran":5,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":9,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"9b4dff79d5e6102c","entry":"apply_rotary_pos_emb","repo":"pppa2019/swie_overmiss_llm4mt","repo_kind":"official","path":"src/trainable_multiL_ins_llama.py","file_url":"https://github.com/pppa2019/swie_overmiss_llm4mt/blob/HEAD/src/trainable_multiL_ins_llama.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9b4dff79d5e6102c"}},{"code_sha256_prefix":"258ba32f9e3eb1d9","entry":"create_prompt","repo":"pppa2019/swie_overmiss_llm4mt","repo_kind":"official","path":"src/inference.py","file_url":"https://github.com/pppa2019/swie_overmiss_llm4mt/blob/HEAD/src/inference.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"258ba32f9e3eb1d9"}},{"code_sha256_prefix":"54458e263ddc9222","entry":"create_prompt","repo":"pppa2019/swie_overmiss_llm4mt","repo_kind":"official","path":"utils/convert_alpaca_to_hf.py","file_url":"https://github.com/pppa2019/swie_overmiss_llm4mt/blob/HEAD/utils/convert_alpaca_to_hf.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"54458e263ddc9222"}},{"code_sha256_prefix":"2a686c7dfc4f6740","entry":"read_input","repo":"pppa2019/swie_overmiss_llm4mt","repo_kind":"official","path":"src/inference.py","file_url":"https://github.com/pppa2019/swie_overmiss_llm4mt/blob/HEAD/src/inference.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2a686c7dfc4f6740"}},{"code_sha256_prefix":"a90bcd4fe56ecc9f","entry":"read_json","repo":"pppa2019/swie_overmiss_llm4mt","repo_kind":"official","path":"utils/convert_alpaca_to_hf.py","file_url":"https://github.com/pppa2019/swie_overmiss_llm4mt/blob/HEAD/utils/convert_alpaca_to_hf.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a90bcd4fe56ecc9f"}},{"code_sha256_prefix":"b99eea6376d1e212","entry":"rotate_half","repo":"pppa2019/swie_overmiss_llm4mt","repo_kind":"official","path":"src/trainable_multiL_ins_llama.py","file_url":"https://github.com/pppa2019/swie_overmiss_llm4mt/blob/HEAD/src/trainable_multiL_ins_llama.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b99eea6376d1e212"}},{"code_sha256_prefix":"8d2f5760a6dbad50","entry":"token_ids2seg_ids","repo":"pppa2019/swie_overmiss_llm4mt","repo_kind":"official","path":"src/trainable_multiL_ins_bloom.py","file_url":"https://github.com/pppa2019/swie_overmiss_llm4mt/blob/HEAD/src/trainable_multiL_ins_bloom.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8d2f5760a6dbad50"}},{"code_sha256_prefix":"8a6488f48ccd46e2","entry":"token_ids2seg_ids","repo":"pppa2019/swie_overmiss_llm4mt","repo_kind":"official","path":"src/trainable_multiL_ins_llama.py","file_url":"https://github.com/pppa2019/swie_overmiss_llm4mt/blob/HEAD/src/trainable_multiL_ins_llama.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8a6488f48ccd46e2"}},{"code_sha256_prefix":"dc9d6423c6ffdfea","entry":"read_instruct","repo":"pppa2019/swie_overmiss_llm4mt","repo_kind":"official","path":"src/inference.py","file_url":"https://github.com/pppa2019/swie_overmiss_llm4mt/blob/HEAD/src/inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dc9d6423c6ffdfea"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}