{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/graph-based-filtering-of-out-of-vocabulary","title":"Graph-based Filtering of Out-of-Vocabulary Words for Encoder-Decoder Models","arxiv_id":"1805.11189","date":"2018-05-28","proceeding":"ACL 2018 7","authors":["Satoru Katsumata","Yukio Matsumura","Hayahide Yamagishi","Mamoru Komachi"],"abstract":"Encoder-decoder models typically only employ words that are frequently used\nin the training corpus to reduce the computational costs and exclude noise.\nHowever, this vocabulary set may still include words that interfere with\nlearning in encoder-decoder models. This paper proposes a method for selecting\nmore suitable words for learning encoders by utilizing not only frequency, but\nalso co-occurrence information, which we capture using the HITS algorithm. We\napply our proposed method to two tasks: machine translation and grammatical\nerror correction. For Japanese-to-English translation, this method achieves a\nBLEU score that is 0.56 points more than that of a baseline. It also\noutperforms the baseline method for English grammatical error correction, with\nan F0.5-measure that is 1.48 points higher.","url_abs":"http://arxiv.org/abs/1805.11189v1","url_pdf":"http://arxiv.org/pdf/1805.11189v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"graph-based-filtering-of-out-of-vocabulary","repo_url":"https://github.com/Katsumata420/HITS_Ranking","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"grammatical-error-correction","task_name":"Grammatical Error Correction"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}