{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/think-globally-embed-locally-locally-linear","title":"Think Globally, Embed Locally --- Locally Linear Meta-embedding of Words","arxiv_id":"1709.06671","date":"2017-09-19","proceeding":null,"authors":["Danushka Bollegala","Kohei Hayashi","Ken-ichi Kawarabayashi"],"abstract":"Distributed word embeddings have shown superior performances in numerous\nNatural Language Processing (NLP) tasks. However, their performances vary\nsignificantly across different tasks, implying that the word embeddings learnt\nby those methods capture complementary aspects of lexical semantics. Therefore,\nwe believe that it is important to combine the existing word embeddings to\nproduce more accurate and complete \\emph{meta-embeddings} of words. For this\npurpose, we propose an unsupervised locally linear meta-embedding learning\nmethod that takes pre-trained word embeddings as the input, and produces more\naccurate meta embeddings. Unlike previously proposed meta-embedding learning\nmethods that learn a global projection over all words in a vocabulary, our\nproposed method is sensitive to the differences in local neighbourhoods of the\nindividual source word embeddings. Moreover, we show that vector concatenation,\na previously proposed highly competitive baseline approach for integrating word\nembeddings, can be derived as a special case of the proposed method.\nExperimental results on semantic similarity, word analogy, relation\nclassification, and short-text classification tasks show that our\nmeta-embeddings to significantly outperform prior methods in several benchmark\ndatasets, establishing a new state of the art for meta-embeddings.","url_abs":"http://arxiv.org/abs/1709.06671v1","url_pdf":"http://arxiv.org/pdf/1709.06671v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"think-globally-embed-locally-locally-linear","repo_url":"https://github.com/Shujian2015/meta-embedding-paper-list","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"relation-classification","task_name":"Relation Classification"},{"task_slug":"semantic-similarity","task_name":"Semantic Similarity"},{"task_slug":"semantic-textual-similarity","task_name":"Semantic Textual Similarity"},{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"word-embeddings","task_name":"Word Embeddings"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1709.06671","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}