{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/word2vec-is-a-special-case-of-kernel","title":"Word2Vec is a special case of Kernel Correspondence Analysis and Kernels for Natural Language Processing","arxiv_id":"1605.05087","date":"2016-05-17","proceeding":null,"authors":["Hirotaka Niitsuma","Minho Lee"],"abstract":"We show that correspondence analysis (CA) is equivalent to defining a Gini\nindex with appropriately scaled one-hot encoding. Using this relation, we\nintroduce a nonlinear kernel extension to CA. This extended CA gives a known\nanalysis for natural language via specialized kernels that use an appropriate\ncontingency table. We propose a semi-supervised CA, which is a special case of\nthe kernel extension to CA. Because CA requires excessive memory if applied to\nnumerous categories, CA has not been used for natural language processing. We\naddress this problem by introducing delayed evaluation to randomized singular\nvalue decomposition. The memory-efficient CA is then applied to a word-vector\nrepresentation task. We propose a tail-cut kernel, which is an extension to the\nskip-gram within the kernel extension to CA. Our tail-cut kernel outperforms\nexisting word-vector representation methods.","url_abs":"http://arxiv.org/abs/1605.05087v3","url_pdf":"http://arxiv.org/pdf/1605.05087v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"word2vec-is-a-special-case-of-kernel","repo_url":"https://github.com/niitsuma/wordca","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}