{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/volt-improving-vocabularization-via-optimal","title":"Vocabulary Learning via Optimal Transport for Neural Machine Translation","arxiv_id":"2012.15671","date":"2020-12-31","proceeding":"ACL 2021 5","authors":["Jingjing Xu","Hao Zhou","Chun Gan","Zaixiang Zheng","Lei LI"],"abstract":"The choice of token vocabulary affects the performance of machine translation. This paper aims to figure out what is a good vocabulary and whether one can find the optimal vocabulary without trial training. To answer these questions, we first provide an alternative understanding of the role of vocabulary from the perspective of information theory. Motivated by this, we formulate the quest of vocabularization -- finding the best token dictionary with a proper size -- as an optimal transport (OT) problem. We propose VOLT, a simple and efficient solution without trial training. Empirical results show that VOLT outperforms widely-used vocabularies in diverse scenarios, including WMT-14 English-German and TED's 52 translation directions. For example, VOLT achieves almost 70% vocabulary size reduction and 0.5 BLEU gain on English-German translation. Also, compared to BPE-search, VOLT reduces the search time from 384 GPU hours to 30 GPU hours on English-German translation. Codes are available at https://github.com/Jingjing-NLP/VOLT .","url_abs":"https://arxiv.org/abs/2012.15671v5","url_pdf":"https://arxiv.org/pdf/2012.15671v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"volt-improving-vocabularization-via-optimal","repo_url":"https://github.com/Jingjing-NLP/VOLT","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":null,"task_name":"GPU"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2012.15671","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2012.15671"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Jingjing-NLP/VOLT","reach":null}],"summary":{"ran_honours":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"8cd046e8cced992a","entry":"build_d_matrix","repo":"Jingjing-NLP/VOLT","repo_kind":"official","path":"ot_run.py","file_url":"https://github.com/Jingjing-NLP/VOLT/blob/HEAD/ot_run.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8cd046e8cced992a"}},{"code_sha256_prefix":"8c4bd92409ca4462","entry":"get_r","repo":"Jingjing-NLP/VOLT","repo_kind":"official","path":"ot_run.py","file_url":"https://github.com/Jingjing-NLP/VOLT/blob/HEAD/ot_run.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8c4bd92409ca4462"}},{"code_sha256_prefix":"ba99fa572ce870a8","entry":"get_total_tokens","repo":"Jingjing-NLP/VOLT","repo_kind":"official","path":"ot_run.py","file_url":"https://github.com/Jingjing-NLP/VOLT/blob/HEAD/ot_run.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ba99fa572ce870a8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}