{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/document-informed-neural-autoregressive-topic-1","title":"Document Informed Neural Autoregressive Topic Models","arxiv_id":"1808.03793","date":"2018-08-11","proceeding":null,"authors":["Pankaj Gupta","Florian Buettner","Hinrich Schütze"],"abstract":"Context information around words helps in determining their actual meaning,\nfor example \"networks\" used in contexts of artificial neural networks or\nbiological neuron networks. Generative topic models infer topic-word\ndistributions, taking no or only little context into account. Here, we extend a\nneural autoregressive topic model to exploit the full context information\naround words in a document in a language modeling fashion. This results in an\nimproved performance in terms of generalization, interpretability and\napplicability. We apply our modeling approach to seven data sets from various\ndomains and demonstrate that our approach consistently outperforms\nstateof-the-art generative topic models. With the learned representations, we\nshow on an average a gain of 9.6% (0.57 Vs 0.52) in precision at retrieval\nfraction 0.02 and 7.2% (0.582 Vs 0.543) in F1 for text categorization.","url_abs":"http://arxiv.org/abs/1808.03793v1","url_pdf":"http://arxiv.org/pdf/1808.03793v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"document-informed-neural-autoregressive-topic-1","repo_url":"https://github.com/pgcool/iDocNADE","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"text-categorization","task_name":"Text Categorization"},{"task_slug":"topic-models","task_name":"Topic Models"}],"methods":[{"method_slug":"interpretability","method_name":"Interpretability"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}