{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/attentive-mimicking-better-word-embeddings-by","title":"Attentive Mimicking: Better Word Embeddings by Attending to Informative Contexts","arxiv_id":"1904.01617","date":"2019-04-02","proceeding":"NAACL 2019 6","authors":["Timo Schick","Hinrich Schütze"],"abstract":"Learning high-quality embeddings for rare words is a hard problem because of\nsparse context information. Mimicking (Pinter et al., 2017) has been proposed\nas a solution: given embeddings learned by a standard algorithm, a model is\nfirst trained to reproduce embeddings of frequent words from their surface form\nand then used to compute embeddings for rare words. In this paper, we introduce\nattentive mimicking: the mimicking model is given access not only to a word's\nsurface form, but also to all available contexts and learns to attend to the\nmost informative and reliable contexts for computing an embedding. In an\nevaluation on four tasks, we show that attentive mimicking outperforms previous\nwork for both rare and medium-frequency words. Thus, compared to previous work,\nattentive mimicking improves embeddings for a much larger part of the\nvocabulary, including the medium-frequency range.","url_abs":"http://arxiv.org/abs/1904.01617v2","url_pdf":"http://arxiv.org/pdf/1904.01617v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"attentive-mimicking-better-word-embeddings-by","repo_url":"https://github.com/timoschick/form-context-model","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"form","task_name":"Form"},{"task_slug":"word-embeddings","task_name":"Word Embeddings"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1904.01617","atlas_url":"https://app.syntology.ai/?focus=1904.01617","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1904.01617"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/timoschick/form-context-model","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9e8effe8d9693e74","entry":"get_logger","repo":"timoschick/form-context-model","repo_kind":"official","path":"fcm/my_log.py","file_url":"https://github.com/timoschick/form-context-model/blob/HEAD/fcm/my_log.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9e8effe8d9693e74"}},{"code_sha256_prefix":"dafac5a5e0891912","entry":"to_n_gram","repo":"timoschick/form-context-model","repo_kind":"official","path":"fcm/utils.py","file_url":"https://github.com/timoschick/form-context-model/blob/HEAD/fcm/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dafac5a5e0891912"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}