{"url":"/method/lda2vec","slug":"lda2vec","name":"lda2vec","full_name":"lda2vec","full_name_withheld":false,"description_markdown":"**lda2vec** builds representations over both words and documents by mixing word2vec’s skipgram architecture with Dirichlet-optimized sparse topic mixtures. \r\n\r\nThe Skipgram Negative-Sampling (SGNS) objective of word2vec is modified to utilize document-wide feature vectors while simultaneously learning continuous document weights loading onto topic vectors. The total loss term $L$ is the sum of the Skipgram Negative Sampling Loss (SGNS) $L^{neg}\\_{ij}$ with the addition of a Dirichlet-likelihood term over document weights, $L\\_{d}$. The loss is conducted using a context vector, $\\overrightarrow{c\\_{j}}$ , pivot word vector $\\overrightarrow{w\\_{j}}$, target word vector $\\overrightarrow{w\\_{i}}$, and negatively-sampled word vector $\\overrightarrow{w\\_{l}}$:\r\n\r\n$$ L = L^{d} + \\Sigma\\_{ij}L^{neg}\\_{ij} $$\r\n\r\n$$L^{neg}\\_{ij} = \\log\\sigma\\left(c\\_{j}\\cdot\\overrightarrow{w\\_{i}}\\right) + \\sum^{n}\\_{l=0}\\sigma\\left(-\\overrightarrow{c\\_{j}}\\cdot\\overrightarrow{w\\_{l}}\\right)$$","description_state":"present","introduced_year":null,"introduced_by":{"title":"Mixing Dirichlet Topic Models and Word Embeddings to Make lda2vec","paper":"/paper/mixing-dirichlet-topic-models-and-word","first_author":"Christopher E Moody","n_authors":1,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/mixing-dirichlet-topic-models-and-word"},"source":{"url":"http://arxiv.org/abs/1605.02019v1","title":"Mixing Dirichlet Topic Models and Word Embeddings to Make lda2vec","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Topic Embeddings","url":"/methods/category/topic-embeddings","pwc_aliases":[]},{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Document Embeddings","url":"/methods/category/document-embeddings","pwc_aliases":[]},{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Static Word Embeddings","url":"/methods/category/static-word-embeddings","pwc_aliases":[]},{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Word Embeddings","url":"/methods/category/word-embeddings","pwc_aliases":[]}],"n_papers_tagged":1,"archive_num_papers":1,"papers_newest_first":[{"paper":"/paper/mixing-dirichlet-topic-models-and-word","title":"Mixing Dirichlet Topic Models and Word Embeddings to Make lda2vec","date":"2016-05-06","arxiv_id":"1605.02019","n_code_links":5,"syntology":{"ran":0,"of":4,"unverified":4,"pointer_only":0}}],"papers_shown":1,"tasks":[{"task":"/task/topic-models","name":"Topic Models","papers":1},{"task":"/task/word-embeddings","name":"Word Embeddings","papers":1}],"tasks_shown":2,"n_tasks":2,"usage_by_year":[{"year":"2016","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/lda2vec"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}