{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/autoencoding-variational-inference-for-topic","title":"Autoencoding Variational Inference For Topic Models","arxiv_id":"1703.01488","date":"2017-03-04","proceeding":null,"authors":["Akash Srivastava","Charles Sutton"],"abstract":"Topic models are one of the most popular methods for learning representations\nof text, but a major challenge is that any change to the topic model requires\nmathematically deriving a new inference algorithm. A promising approach to\naddress this problem is autoencoding variational Bayes (AEVB), but it has\nproven diffi- cult to apply to topic models in practice. We present what is to\nour knowledge the first effective AEVB based inference method for latent\nDirichlet allocation (LDA), which we call Autoencoded Variational Inference For\nTopic Model (AVITM). This model tackles the problems caused for AEVB by the\nDirichlet prior and by component collapsing. We find that AVITM matches\ntraditional methods in accuracy with much better inference time. Indeed,\nbecause of the inference network, we find that it is unnecessary to pay the\ncomputational cost of running variational optimization on test data. Because\nAVITM is black box, it is readily applied to new topic models. As a dramatic\nillustration of this, we present a new topic model called ProdLDA, that\nreplaces the mixture model in LDA with a product of experts. By changing only\none line of code from LDA, we find that ProdLDA yields much more interpretable\ntopics, even if LDA is trained via collapsed Gibbs sampling.","url_abs":"http://arxiv.org/abs/1703.01488v1","url_pdf":"http://arxiv.org/pdf/1703.01488v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"autoencoding-variational-inference-for-topic","repo_url":"https://github.com/akashgit/autoencoding_vi_for_topic_models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"autoencoding-variational-inference-for-topic","repo_url":"https://github.com/is0383kk/Dirichlet_VAE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"autoencoding-variational-inference-for-topic","repo_url":"https://github.com/mind-Lab/octis","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"autoencoding-variational-inference-for-topic","repo_url":"https://github.com/shining-spring/nvlda","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"autoencoding-variational-inference-for-topic","repo_url":"https://github.com/vlukiyanov/pt-avitm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"autoencoding-variational-inference-for-topic","repo_url":"https://github.com/yjxiao/ProdLDA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"topic-models","task_name":"Topic Models"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[{"method_slug":"lda","method_name":"LDA"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/topic-models-on-20newsgroups","task":"Topic Models","dataset":"20NewsGroups","model":"ProdLDA","rank_in_archive_order":6,"of":6,"metrics":{"C_v":"0.35"},"uses_additional_data":false},{"leaderboard":"/sota/topic-models-on-ag-news","task":"Topic Models","dataset":"AG News","model":"ProdLDA","rank_in_archive_order":6,"of":6,"metrics":{"C_v":"0.32","NPMI":"-0.22"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1703.01488","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1703.01488"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/akashgit/autoencoding_vi_for_topic_models","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shining-spring/nvlda","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vlukiyanov/pt-avitm","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/is0383kk/Dirichlet_VAE","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yjxiao/ProdLDA","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mind-Lab/octis","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"unverified":7},"by_repo_kind":{"listed":{"samples":7,"ran":0,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"c7a758418e234da7","entry":"read_data","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"c7a758418e234da7"}},{"code_sha256_prefix":"8b988d62007ee9ce","entry":"decoder","repo":"vlukiyanov/pt-avitm","repo_kind":"listed","path":"ptavitm/vae.py","file_url":"https://github.com/vlukiyanov/pt-avitm/blob/HEAD/ptavitm/vae.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8b988d62007ee9ce"}},{"code_sha256_prefix":"200ab38e5c9b742f","entry":"encoder","repo":"vlukiyanov/pt-avitm","repo_kind":"listed","path":"ptavitm/vae.py","file_url":"https://github.com/vlukiyanov/pt-avitm/blob/HEAD/ptavitm/vae.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"200ab38e5c9b742f"}},{"code_sha256_prefix":"db73bb820e56e989","entry":"log_dir_init","repo":"akashgit/autoencoding_vi_for_topic_models","repo_kind":"listed","path":"models/nvlda.py","file_url":"https://github.com/akashgit/autoencoding_vi_for_topic_models/blob/HEAD/models/nvlda.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"db73bb820e56e989"}},{"code_sha256_prefix":"4b4bc8c55147a8c5","entry":"perplexity","repo":"vlukiyanov/pt-avitm","repo_kind":"listed","path":"ptavitm/model.py","file_url":"https://github.com/vlukiyanov/pt-avitm/blob/HEAD/ptavitm/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4b4bc8c55147a8c5"}},{"code_sha256_prefix":"c07b9b907915c094","entry":"predict","repo":"vlukiyanov/pt-avitm","repo_kind":"listed","path":"ptavitm/model.py","file_url":"https://github.com/vlukiyanov/pt-avitm/blob/HEAD/ptavitm/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c07b9b907915c094"}},{"code_sha256_prefix":"90a17b6a9a9e2d19","entry":"prior","repo":"vlukiyanov/pt-avitm","repo_kind":"listed","path":"ptavitm/vae.py","file_url":"https://github.com/vlukiyanov/pt-avitm/blob/HEAD/ptavitm/vae.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"90a17b6a9a9e2d19"}},{"code_sha256_prefix":"ce2f744c3f060169","entry":"xavier_init","repo":"akashgit/autoencoding_vi_for_topic_models","repo_kind":"listed","path":"models/nvlda.py","file_url":"https://github.com/akashgit/autoencoding_vi_for_topic_models/blob/HEAD/models/nvlda.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce2f744c3f060169"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}