{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gating-mechanisms-for-combining-character-and","title":"Gating Mechanisms for Combining Character and Word-level Word Representations: An Empirical Study","arxiv_id":"1904.05584","date":"2019-04-11","proceeding":"NAACL 2019 6","authors":["Jorge A. Balazs","Yutaka Matsuo"],"abstract":"In this paper we study how different ways of combining character and\nword-level representations affect the quality of both final word and sentence\nrepresentations. We provide strong empirical evidence that modeling characters\nimproves the learned representations at the word and sentence levels, and that\ndoing so is particularly useful when representing less frequent words. We\nfurther show that a feature-wise sigmoid gating mechanism is a robust method\nfor creating representations that encode semantic similarity, as it performed\nreasonably well in several word similarity datasets. Finally, our findings\nsuggest that properly capturing semantic similarity at the word level does not\nconsistently yield improved performance in downstream sentence-level tasks. Our\ncode is available at https://github.com/jabalazs/gating","url_abs":"http://arxiv.org/abs/1904.05584v1","url_pdf":"http://arxiv.org/pdf/1904.05584v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gating-mechanisms-for-combining-character-and","repo_url":"https://github.com/jabalazs/gating","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"semantic-similarity","task_name":"Semantic Similarity"},{"task_slug":"semantic-textual-similarity","task_name":"Semantic Textual Similarity"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"word-similarity","task_name":"Word Similarity"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}