{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/word2vec-parameter-learning-explained","title":"word2vec Parameter Learning Explained","arxiv_id":"1411.2738","date":"2014-11-11","proceeding":null,"authors":["Xin Rong"],"abstract":"The word2vec model and application by Mikolov et al. have attracted a great\namount of attention in recent two years. The vector representations of words\nlearned by word2vec models have been shown to carry semantic meanings and are\nuseful in various NLP tasks. As an increasing number of researchers would like\nto experiment with word2vec or similar techniques, I notice that there lacks a\nmaterial that comprehensively explains the parameter learning process of word\nembedding models in details, thus preventing researchers that are non-experts\nin neural networks from understanding the working mechanism of such models.\n  This note provides detailed derivations and explanations of the parameter\nupdate equations of the word2vec models, including the original continuous\nbag-of-word (CBOW) and skip-gram (SG) models, as well as advanced optimization\ntechniques, including hierarchical softmax and negative sampling. Intuitive\ninterpretations of the gradient equations are also provided alongside\nmathematical derivations.\n  In the appendix, a review on the basics of neuron networks and\nbackpropagation is provided. I also created an interactive demo, wevi, to\nfacilitate the intuitive understanding of the model.","url_abs":"http://arxiv.org/abs/1411.2738v4","url_pdf":"http://arxiv.org/pdf/1411.2738v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"word2vec-parameter-learning-explained","repo_url":"https://github.com/ronxin/wevi","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"word2vec-parameter-learning-explained","repo_url":"https://github.com/LouisTernon/NLP","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"word2vec-parameter-learning-explained","repo_url":"https://github.com/fabiomercorio/GraphDBLP","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"word2vec-parameter-learning-explained","repo_url":"https://github.com/gymk/ANLP","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"word2vec-parameter-learning-explained","repo_url":"https://github.com/linghduoduo/NLP","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"word2vec-parameter-learning-explained","repo_url":"https://github.com/mayank2498/Skip-Gram-model-using-numpy","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"word2vec-parameter-learning-explained","repo_url":"https://github.com/mingdaoyang/workshop","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"word2vec-parameter-learning-explained","repo_url":"https://github.com/lvyufeng/mindspore-nlp-tutorial","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"hierarchical-softmax","method_name":"Hierarchical Softmax"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1411.2738","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1411.2738"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mingdaoyang/workshop","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mayank2498/Skip-Gram-model-using-numpy","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/linghduoduo/NLP","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/LouisTernon/NLP","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lvyufeng/mindspore-nlp-tutorial","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ronxin/wevi","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/fabiomercorio/GraphDBLP","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gymk/ANLP","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":2,"ran_honours":2,"unverified":5},"by_repo_kind":{"listed":{"samples":7,"ran":2,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"24b8995c9efdcdb7","entry":"loadPairs","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"24b8995c9efdcdb7"}},{"code_sha256_prefix":"78fb2b422c62f1e5","entry":"prepare_data_for_training","repo":"mayank2498/Skip-Gram-model-using-numpy","repo_kind":"listed","path":"word2vec.py","file_url":"https://github.com/mayank2498/Skip-Gram-model-using-numpy/blob/HEAD/word2vec.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"78fb2b422c62f1e5"}},{"code_sha256_prefix":"c0cf1196424d4f71","entry":"softmax","repo":"mayank2498/Skip-Gram-model-using-numpy","repo_kind":"listed","path":"word2vec.py","file_url":"https://github.com/mayank2498/Skip-Gram-model-using-numpy/blob/HEAD/word2vec.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c0cf1196424d4f71"}},{"code_sha256_prefix":"375693019fa03226","entry":"text2sentences","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"375693019fa03226"}},{"code_sha256_prefix":"d454996ae5afa9b9","entry":"H_approx","repo":"gymk/ANLP","repo_kind":"listed","path":"zipfs_law-2.py","file_url":"https://github.com/gymk/ANLP/blob/HEAD/zipfs_law-2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d454996ae5afa9b9"}},{"code_sha256_prefix":"d1d3a336b99abbf2","entry":"cosine_similarity","repo":"gymk/ANLP","repo_kind":"listed","path":"Week2-Quiz-4.py","file_url":"https://github.com/gymk/ANLP/blob/HEAD/Week2-Quiz-4.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d1d3a336b99abbf2"}},{"code_sha256_prefix":"ca439c565aba5b68","entry":"getFingerPrintOnCount","repo":"gymk/ANLP","repo_kind":"listed","path":"Week2-Quiz-5to7.py","file_url":"https://github.com/gymk/ANLP/blob/HEAD/Week2-Quiz-5to7.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ca439c565aba5b68"}},{"code_sha256_prefix":"61583f325997c9a0","entry":"getScore","repo":"gymk/ANLP","repo_kind":"listed","path":"Week10/Q9.py","file_url":"https://github.com/gymk/ANLP/blob/HEAD/Week10/Q9.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61583f325997c9a0"}},{"code_sha256_prefix":"d543f3d5ca3a0afa","entry":"getTanimotoCoefficient","repo":"gymk/ANLP","repo_kind":"listed","path":"Week2-Quiz-5to7.py","file_url":"https://github.com/gymk/ANLP/blob/HEAD/Week2-Quiz-5to7.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d543f3d5ca3a0afa"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}