{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/representing-mixtures-of-word-embeddings-with-1","title":"Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings","arxiv_id":"2203.01570","date":"2022-03-03","proceeding":"ICLR 2022 4","authors":["Dongsheng Wang","Dandan Guo","He Zhao","Huangjie Zheng","Korawat Tanwisuth","Bo Chen","Mingyuan Zhou"],"abstract":"A topic model is often formulated as a generative model that explains how each word of a document is generated given a set of topics and document-specific topic proportions. It is focused on capturing the word co-occurrences in a document and hence often suffers from poor performance in analyzing short documents. In addition, its parameter estimation often relies on approximate posterior inference that is either not scalable or suffers from large approximation error. This paper introduces a new topic-modeling framework where each document is viewed as a set of word embedding vectors and each topic is modeled as an embedding vector in the same embedding space. Embedding the words and topics in the same vector space, we define a method to measure the semantic difference between the embedding vectors of the words of a document and these of the topics, and optimize the topic embeddings to minimize the expected difference over all documents. Experiments on text analysis demonstrate that the proposed method, which is amenable to mini-batch stochastic gradient descent based optimization and hence scalable to big corpora, provides competitive performance in discovering more coherent and diverse topics and extracting better document representations.","url_abs":"https://arxiv.org/abs/2203.01570v2","url_pdf":"https://arxiv.org/pdf/2203.01570v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"representing-mixtures-of-word-embeddings-with-1","repo_url":"https://github.com/BoChenGroup/WeTe","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"representing-mixtures-of-word-embeddings-with-1","repo_url":"https://github.com/wds2014/wete","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"word-embeddings","task_name":"Word Embeddings"},{"task_slug":"parameter-estimation","task_name":"parameter estimation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2203.01570","atlas_url":"https://app.syntology.ai/?focus=2203.01570","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.01570"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/BoChenGroup/WeTe","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wds2014/wete","reach":null}],"summary":{"ran":2,"ran_honours":2,"unverified":2},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1},"listed":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d81cf76f0aaa6877","entry":"Infer_Net","repo":"wds2014/wete","repo_kind":"listed","path":"model.py","file_url":"https://github.com/wds2014/wete/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d81cf76f0aaa6877"}},{"code_sha256_prefix":"cafdd7fc481eccfd","entry":"WeTe","repo":"BoChenGroup/WeTe","repo_kind":"official","path":"model.py","file_url":"https://github.com/BoChenGroup/WeTe/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cafdd7fc481eccfd"}},{"code_sha256_prefix":"8e221d38153ae3cb","entry":"normalization","repo":"bochengroup/wete","repo_kind":"official","path":"cluster_clc.py","file_url":"https://github.com/bochengroup/wete/blob/HEAD/cluster_clc.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8e221d38153ae3cb"}},{"code_sha256_prefix":"a06b554c49675416","entry":"standardization","repo":"bochengroup/wete","repo_kind":"official","path":"cluster_clc.py","file_url":"https://github.com/bochengroup/wete/blob/HEAD/cluster_clc.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a06b554c49675416"}},{"code_sha256_prefix":"8766444242c5df71","entry":"WeTe","repo":"wds2014/wete","repo_kind":"listed","path":"model.py","file_url":"https://github.com/wds2014/wete/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8766444242c5df71"}},{"code_sha256_prefix":"621585535245758a","entry":"purity","repo":"bochengroup/wete","repo_kind":"official","path":"cluster_clc.py","file_url":"https://github.com/bochengroup/wete/blob/HEAD/cluster_clc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"621585535245758a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}