{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sparsity-makes-sense-word-sense","title":"Sparsity Makes Sense: Word Sense Disambiguation Using Sparse Contextualized Word Representations","arxiv_id":null,"date":"2020-11-01","proceeding":"EMNLP 2020 11","authors":["G{\\'a}bor Berend"],"abstract":"In this paper, we demonstrate that by utilizing sparse word representations, it becomes possible to surpass the results of more complex task-specific models on the task of fine-grained all-words word sense disambiguation. Our proposed algorithm relies on an overcomplete set of semantic basis vectors that allows us to obtain sparse contextualized word representations. We introduce such an information theory-inspired synset representation based on the co-occurrence of word senses and non-zero coordinates for word forms which allows us to achieve an aggregated F-score of 78.8 over a combination of five standard word sense disambiguating benchmark datasets. We also demonstrate the general applicability of our proposed framework by evaluating it towards part-of-speech tagging on four different treebanks. Our results indicate a significant improvement over the application of the dense word representations.","url_abs":"https://aclanthology.org/2020.emnlp-main.683","url_pdf":"https://aclanthology.org/2020.emnlp-main.683.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"part-of-speech-tagging","task_name":"Part-Of-Speech Tagging"},{"task_slug":"word-sense-disambiguation","task_name":"Word Sense Disambiguation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/word-sense-disambiguation-on-supervised","task":"Word Sense Disambiguation","dataset":"Supervised:","model":"SparseLMMS+WNGC","rank_in_archive_order":10,"of":27,"metrics":{"SemEval 2007":"73.0","SemEval 2013":"79.4","SemEval 2015":"81.3","Senseval 2":"79.6","Senseval 3":"77.3"},"uses_additional_data":false},{"leaderboard":"/sota/word-sense-disambiguation-on-supervised","task":"Word Sense Disambiguation","dataset":"Supervised:","model":"SparseLMMS","rank_in_archive_order":14,"of":27,"metrics":{"SemEval 2007":"68.8","SemEval 2013":"76.1","SemEval 2015":"77.5","Senseval 2":"77.9","Senseval 3":"77.8"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}