{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/registering-source-tokens-to-target-language","title":"Registering Source Tokens to Target Language Spaces in Multilingual Neural Machine Translation","arxiv_id":"2501.02979","date":"2025-01-06","proceeding":null,"authors":["Zhi Qu","Yiran Wang","Jiannan Mao","Chenchen Ding","Hideki Tanaka","Masao Utiyama","Taro Watanabe"],"abstract":"The multilingual neural machine translation (MNMT) enables arbitrary translations across multiple languages by training a model with limited parameters using parallel data only. However, the performance of such MNMT models still lags behind that of large language models (LLMs), limiting their practicality. In this work, we address this limitation by introducing registering to achieve the new state-of-the-art of decoder-only MNMT models. Specifically, we insert a set of artificial tokens specifying the target language, called registers, into the input sequence between the source and target tokens. By modifying the attention mask, the target token generation only pays attention to the activation of registers, representing the source tokens in the target language space. Experiments on EC-40, a large-scale benchmark, show that our method outperforms related methods driven by optimizing multilingual representations. We further scale up and collect 9.3 billion sentence pairs across 24 languages from public datasets to pre-train two models, namely MITRE (multilingual translation with registers). One of them, MITRE-913M, outperforms NLLB-3.3B, achieves comparable performance with commercial LLMs, and shows strong adaptability in fine-tuning. Finally, we open-source our models to facilitate further research and development in MNMT: https://github.com/zhiqu22/mitre.","url_abs":"https://arxiv.org/abs/2501.02979v1","url_pdf":"https://arxiv.org/pdf/2501.02979v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"registering-source-tokens-to-target-language","repo_url":"https://github.com/zhiqu22/mitre","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"registering-source-tokens-to-target-language","repo_url":"https://huggingface.co/naist-nlp/mitre_466m","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null},{"paper_slug":"registering-source-tokens-to-target-language","repo_url":"https://huggingface.co/naist-nlp/mitre_913m","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"translation","task_name":"Translation"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"set","method_name":"SET"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2501.02979","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2501.02979"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/zhiqu22/mitre","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c35e87d7ba792212","entry":"count_results_by_class","repo":"zhiqu22/mitre","repo_kind":"official","path":"ec_40_scripts/evaluation/make_table.py","file_url":"https://github.com/zhiqu22/mitre/blob/HEAD/ec_40_scripts/evaluation/make_table.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c35e87d7ba792212"}},{"code_sha256_prefix":"13cfe21ed5a7b812","entry":"module_name_fordropout","repo":"zhiqu22/mitre","repo_kind":"official","path":"mitre/models/transformer_decoder_register.py","file_url":"https://github.com/zhiqu22/mitre/blob/HEAD/mitre/models/transformer_decoder_register.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"13cfe21ed5a7b812"}},{"code_sha256_prefix":"ae285f7d6c2a685e","entry":"module_name_fordropout","repo":"zhiqu22/mitre","repo_kind":"official","path":"mitre/models/transformer_encoder_register.py","file_url":"https://github.com/zhiqu22/mitre/blob/HEAD/mitre/models/transformer_encoder_register.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ae285f7d6c2a685e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}