{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/offline-bilingual-word-vectors-orthogonal","title":"Offline bilingual word vectors, orthogonal transformations and the inverted softmax","arxiv_id":"1702.03859","date":"2017-02-13","proceeding":null,"authors":["Samuel L. Smith","David H. P. Turban","Steven Hamblin","Nils Y. Hammerla"],"abstract":"Usually bilingual word vectors are trained \"online\". Mikolov et al. showed\nthey can also be found \"offline\", whereby two pre-trained embeddings are\naligned with a linear transformation, using dictionaries compiled from expert\nknowledge. In this work, we prove that the linear transformation between two\nspaces should be orthogonal. This transformation can be obtained using the\nsingular value decomposition. We introduce a novel \"inverted softmax\" for\nidentifying translation pairs, with which we improve the precision @1 of\nMikolov's original mapping from 34% to 43%, when translating a test set\ncomposed of both common and rare English words into Italian. Orthogonal\ntransformations are more robust to noise, enabling us to learn the\ntransformation without expert bilingual signal by constructing a\n\"pseudo-dictionary\" from the identical character strings which appear in both\nlanguages, achieving 40% precision on the same test set. Finally, we extend our\nmethod to retrieve the true translations of English sentences from a corpus of\n200k Italian sentences with a precision @1 of 68%.","url_abs":"http://arxiv.org/abs/1702.03859v1","url_pdf":"http://arxiv.org/pdf/1702.03859v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"offline-bilingual-word-vectors-orthogonal","repo_url":"https://github.com/Babylonpartners/fastText_multilingual","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"paper_slug":"offline-bilingual-word-vectors-orthogonal","repo_url":"https://github.com/babylonhealth/fastText_multilingual","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"paper_slug":"offline-bilingual-word-vectors-orthogonal","repo_url":"https://github.com/baidu-research/HNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"offline-bilingual-word-vectors-orthogonal","repo_url":"https://github.com/facebookresearch/MUSE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"offline-bilingual-word-vectors-orthogonal","repo_url":"https://github.com/jiajunhua/facebookresearch-MUSE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"offline-bilingual-word-vectors-orthogonal","repo_url":"https://github.com/styx97/PolarizationAAAI2021","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1702.03859","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}