{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/discovering-discrete-latent-topics-with","title":"Discovering Discrete Latent Topics with Neural Variational Inference","arxiv_id":"1706.00359","date":"2017-06-01","proceeding":"ICML 2017 8","authors":["Yishu Miao","Edward Grefenstette","Phil Blunsom"],"abstract":"Topic models have been widely explored as probabilistic generative models of\ndocuments. Traditional inference methods have sought closed-form derivations\nfor updating the models, however as the expressiveness of these models grows,\nso does the difficulty of performing fast and accurate inference over their\nparameters. This paper presents alternative neural approaches to topic\nmodelling by providing parameterisable distributions over topics which permit\ntraining by backpropagation in the framework of neural variational inference.\nIn addition, with the help of a stick-breaking construction, we propose a\nrecurrent network that is able to discover a notionally unbounded number of\ntopics, analogous to Bayesian non-parametric topic models. Experimental results\non the MXM Song Lyrics, 20NewsGroups and Reuters News datasets demonstrate the\neffectiveness and efficiency of these neural topic models.","url_abs":"http://arxiv.org/abs/1706.00359v2","url_pdf":"http://arxiv.org/pdf/1706.00359v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"discovering-discrete-latent-topics-with","repo_url":"https://github.com/YongfeiYan/Neural-Document-Modeling","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"topic-models","task_name":"Topic Models"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/topic-models-on-20newsgroups","task":"Topic Models","dataset":"20NewsGroups","model":"GSM","rank_in_archive_order":2,"of":6,"metrics":{"C_v":"0.55"},"uses_additional_data":false},{"leaderboard":"/sota/topic-models-on-ag-news","task":"Topic Models","dataset":"AG News","model":"GSM","rank_in_archive_order":3,"of":6,"metrics":{"C_v":"0.41","NPMI":"0.03"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1706.00359","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}