{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/word2vec-optimal-hyper-parameters-and-their","title":"Word2Vec: Optimal Hyper-Parameters and Their Impact on NLP Downstream Tasks","arxiv_id":"2003.11645","date":"2020-03-23","proceeding":null,"authors":["Tosin P. Adewumi","Foteini Liwicki","Marcus Liwicki"],"abstract":"Word2Vec is a prominent model for natural language processing (NLP) tasks. Similar inspiration is found in distributed embeddings for new state-of-the-art (SotA) deep neural networks. However, wrong combination of hyper-parameters can produce poor quality vectors. The objective of this work is to empirically show optimal combination of hyper-parameters exists and evaluate various combinations. We compare them with the released, pre-trained original word2vec model. Both intrinsic and extrinsic (downstream) evaluations, including named entity recognition (NER) and sentiment analysis (SA) were carried out. The downstream tasks reveal that the best model is usually task-specific, high analogy scores don't necessarily correlate positively with F1 scores and the same applies to focus on data alone. Increasing vector dimension size after a point leads to poor quality or performance. If ethical considerations to save time, energy and the environment are made, then reasonably smaller corpora may do just as well or even better in some cases. Besides, using a small corpus, we obtain better human-assigned WordSim scores, corresponding Spearman correlation and better downstream performances (with significance tests) compared to the original model, trained on 100 billion-word corpus.","url_abs":"https://arxiv.org/abs/2003.11645v3","url_pdf":"https://arxiv.org/pdf/2003.11645v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"word2vec-optimal-hyper-parameters-and-their","repo_url":"https://github.com/tosingithub/sdesk","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"word2vec-optimal-hyper-parameters-and-their","repo_url":"https://github.com/phnk/D7047E","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"cg","task_name":"NER"},{"task_slug":"named-entity-recognition-1","task_name":"Named Entity Recognition"},{"task_slug":"named-entity-recognition-ner","task_name":"Named Entity Recognition (NER)"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"named-entity-recognition","task_name":"named-entity-recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2003.11645","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2003.11645"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tosingithub/sdesk","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/phnk/D7047E","reach":{"status":"ok"}}],"summary":{"ran_fixture":1,"ran_honours":2,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"4f181f07e783f657","entry":"pad_seq","repo":"tosingithub/sdesk","repo_kind":"official","path":"lstm_ner.py","file_url":"https://github.com/tosingithub/sdesk/blob/HEAD/lstm_ner.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4f181f07e783f657"}},{"code_sha256_prefix":"c83f7b826084dbc6","entry":"prepare_sequence","repo":"tosingithub/sdesk","repo_kind":"official","path":"lstm_ner.py","file_url":"https://github.com/tosingithub/sdesk/blob/HEAD/lstm_ner.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c83f7b826084dbc6"}},{"code_sha256_prefix":"568004bc3f4f896e","entry":"preprocess_pandas","repo":"tosingithub/sdesk","repo_kind":"official","path":"lstm_sa.py","file_url":"https://github.com/tosingithub/sdesk/blob/HEAD/lstm_sa.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"568004bc3f4f896e"}},{"code_sha256_prefix":"8b05b58b2902d098","entry":"seq_to_ix","repo":"tosingithub/sdesk","repo_kind":"official","path":"lstm_sa.py","file_url":"https://github.com/tosingithub/sdesk/blob/HEAD/lstm_sa.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8b05b58b2902d098"}},{"code_sha256_prefix":"5163d159e92415a1","entry":"encode_sents","repo":"tosingithub/sdesk","repo_kind":"official","path":"lstm_sa.py","file_url":"https://github.com/tosingithub/sdesk/blob/HEAD/lstm_sa.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5163d159e92415a1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}