{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/neural-variational-inference-for-text","title":"Neural Variational Inference for Text Processing","arxiv_id":"1511.06038","date":"2015-11-19","proceeding":null,"authors":["Yishu Miao","Lei Yu","Phil Blunsom"],"abstract":"Recent advances in neural variational inference have spawned a renaissance in\ndeep latent variable models. In this paper we introduce a generic variational\ninference framework for generative and conditional models of text. While\ntraditional variational methods derive an analytic approximation for the\nintractable distributions over latent variables, here we construct an inference\nnetwork conditioned on the discrete text input to provide the variational\ndistribution. We validate this framework on two very different text modelling\napplications, generative document modelling and supervised question answering.\nOur neural variational document model combines a continuous stochastic document\nrepresentation with a bag-of-words generative model and achieves the lowest\nreported perplexities on two standard test corpora. The neural answer selection\nmodel employs a stochastic representation layer within an attention mechanism\nto extract the semantics between a question and answer pair. On two question\nanswering benchmarks this model exceeds all previous published benchmarks.","url_abs":"http://arxiv.org/abs/1511.06038v4","url_pdf":"http://arxiv.org/pdf/1511.06038v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"neural-variational-inference-for-text","repo_url":"https://github.com/YongfeiYan/Neural-Document-Modeling","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"neural-variational-inference-for-text","repo_url":"https://github.com/carpedm20/variational-text-tensorflow","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"neural-variational-inference-for-text","repo_url":"https://github.com/jainshobhit/Variational-Autoencoder","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"neural-variational-inference-for-text","repo_url":"https://github.com/jiacheng-xu/vmf_vae_nlp","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"neural-variational-inference-for-text","repo_url":"https://github.com/shining-spring/nvlda","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"neural-variational-inference-for-text","repo_url":"https://github.com/ysmiao/nvdm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"answer-selection","task_name":"Answer Selection"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"topic-models","task_name":"Topic Models"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/question-answering-on-qasent","task":"Question Answering","dataset":"QASent","model":"Attentive LSTM","rank_in_archive_order":1,"of":7,"metrics":{"MAP":"0.7339","MRR":"0.8117"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-qasent","task":"Question Answering","dataset":"QASent","model":"LSTM (lexical overlap + dist output)","rank_in_archive_order":2,"of":7,"metrics":{"MAP":"0.7228","MRR":"0.7986"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-qasent","task":"Question Answering","dataset":"QASent","model":"LSTM","rank_in_archive_order":5,"of":7,"metrics":{"MAP":"0.6436","MRR":"0.7235"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-wikiqa","task":"Question Answering","dataset":"WikiQA","model":"Attentive LSTM","rank_in_archive_order":15,"of":25,"metrics":{"MAP":"0.6886","MRR":"0.7069"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-wikiqa","task":"Question Answering","dataset":"WikiQA","model":"LSTM (lexical overlap + dist output)","rank_in_archive_order":17,"of":25,"metrics":{"MAP":"0.682","MRR":"0.6988"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-wikiqa","task":"Question Answering","dataset":"WikiQA","model":"LSTM","rank_in_archive_order":20,"of":25,"metrics":{"MAP":"0.6552","MRR":"0.6747"},"uses_additional_data":false},{"leaderboard":"/sota/topic-models-on-20-newsgroups","task":"Topic Models","dataset":"20 Newsgroups","model":"NVDM","rank_in_archive_order":2,"of":2,"metrics":{"Test perplexity":"836"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1511.06038","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1511.06038"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ysmiao/nvdm","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shining-spring/nvlda","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/carpedm20/variational-text-tensorflow","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jainshobhit/Variational-Autoencoder","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/YongfeiYan/Neural-Document-Modeling","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jiacheng-xu/vmf_vae_nlp","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"listed":{"samples":4,"ran":1,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"c7a758418e234da7","entry":"read_data","repo":"shining-spring/nvlda","repo_kind":"listed","path":"examples/prepare_20news_data.py","file_url":"https://github.com/shining-spring/nvlda/blob/HEAD/examples/prepare_20news_data.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c7a758418e234da7"}},{"code_sha256_prefix":"1f8d62e9dcc2ed5b","entry":"get_neighbors","repo":"carpedm20/variational-text-tensorflow","repo_kind":"listed","path":"toy_generator.py","file_url":"https://github.com/carpedm20/variational-text-tensorflow/blob/HEAD/toy_generator.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1f8d62e9dcc2ed5b"}},{"code_sha256_prefix":"96027a5753a7ea3c","entry":"load","repo":"carpedm20/variational-text-tensorflow","repo_kind":"listed","path":"batch_loader.py","file_url":"https://github.com/carpedm20/variational-text-tensorflow/blob/HEAD/batch_loader.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"96027a5753a7ea3c"}},{"code_sha256_prefix":"ebd45c240000a52c","entry":"load_npy","repo":"carpedm20/variational-text-tensorflow","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/carpedm20/variational-text-tensorflow/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ebd45c240000a52c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}