{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/parameter-sharing-methods-for-multilingual","title":"Parameter Sharing Methods for Multilingual Self-Attentional Translation Models","arxiv_id":"1809.00252","date":"2018-09-01","proceeding":"WS 2018 10","authors":["Devendra Singh Sachan","Graham Neubig"],"abstract":"In multilingual neural machine translation, it has been shown that sharing a\nsingle translation model between multiple languages can achieve competitive\nperformance, sometimes even leading to performance gains over bilingually\ntrained models. However, these improvements are not uniform; often multilingual\nparameter sharing results in a decrease in accuracy due to translation models\nnot being able to accommodate different languages in their limited parameter\nspace. In this work, we examine parameter sharing techniques that strike a\nhappy medium between full sharing and individual training, specifically\nfocusing on the self-attentional Transformer model. We find that the full\nparameter sharing approach leads to increases in BLEU scores mainly when the\ntarget languages are from a similar language family. However, even in the case\nwhere target languages are from different families where full parameter sharing\nleads to a noticeable drop in BLEU scores, our proposed methods for partial\nsharing of parameters can lead to substantial improvements in translation\naccuracy.","url_abs":"http://arxiv.org/abs/1809.00252v2","url_pdf":"http://arxiv.org/pdf/1809.00252v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"parameter-sharing-methods-for-multilingual","repo_url":"https://github.com/DevSinghSachan/multilingual_nmt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"translation","task_name":"Translation"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1809.00252","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}