{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/riemannian-optimization-for-skip-gram","title":"Riemannian Optimization for Skip-Gram Negative Sampling","arxiv_id":"1704.08059","date":"2017-04-26","proceeding":"ACL 2017 7","authors":["Alexander Fonarev","Oleksii Hrinchuk","Gleb Gusev","Pavel Serdyukov","Ivan Oseledets"],"abstract":"Skip-Gram Negative Sampling (SGNS) word embedding model, well known by its\nimplementation in \"word2vec\" software, is usually optimized by stochastic\ngradient descent. However, the optimization of SGNS objective can be viewed as\na problem of searching for a good matrix with the low-rank constraint. The most\nstandard way to solve this type of problems is to apply Riemannian optimization\nframework to optimize the SGNS objective over the manifold of required low-rank\nmatrices. In this paper, we propose an algorithm that optimizes SGNS objective\nusing Riemannian optimization and demonstrates its superiority over popular\ncompetitors, such as the original method to train SGNS and SVD over SPPMI\nmatrix.","url_abs":"http://arxiv.org/abs/1704.08059v1","url_pdf":"http://arxiv.org/pdf/1704.08059v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"riemannian-optimization-for-skip-gram","repo_url":"https://github.com/AlexGrinch/ro_sgns","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"riemannian-optimization","task_name":"Riemannian optimization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}