{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/topicrnn-a-recurrent-neural-network-with-long","title":"TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency","arxiv_id":"1611.01702","date":"2016-11-05","proceeding":null,"authors":["Adji B. Dieng","Chong Wang","Jianfeng Gao","John Paisley"],"abstract":"In this paper, we propose TopicRNN, a recurrent neural network (RNN)-based\nlanguage model designed to directly capture the global semantic meaning\nrelating words in a document via latent topics. Because of their sequential\nnature, RNNs are good at capturing the local structure of a word sequence -\nboth semantic and syntactic - but might face difficulty remembering long-range\ndependencies. Intuitively, these long-range dependencies are of semantic\nnature. In contrast, latent topic models are able to capture the global\nunderlying semantic structure of a document but do not account for word\nordering. The proposed TopicRNN model integrates the merits of RNNs and latent\ntopic models: it captures local (syntactic) dependencies using an RNN and\nglobal (semantic) dependencies using latent topics. Unlike previous work on\ncontextual RNN language modeling, our model is learned end-to-end. Empirical\nresults on word prediction show that TopicRNN outperforms existing contextual\nRNN baselines. In addition, TopicRNN can be used as an unsupervised feature\nextractor for documents. We do this for sentiment analysis on the IMDB movie\nreview dataset and report an error rate of $6.28\\%$. This is comparable to the\nstate-of-the-art $5.91\\%$ resulting from a semi-supervised approach. Finally,\nTopicRNN also yields sensible topics, making it a useful alternative to\ndocument models such as latent Dirichlet allocation.","url_abs":"http://arxiv.org/abs/1611.01702v2","url_pdf":"http://arxiv.org/pdf/1611.01702v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"topicrnn-a-recurrent-neural-network-with-long","repo_url":"https://github.com/dangitstam/topic-rnn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"topic-models","task_name":"Topic Models"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1611.01702","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1611.01702"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dangitstam/topic-rnn","reach":null}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b156d9c122f71055","entry":"directory_to_jsons","repo":"dangitstam/topic-rnn","repo_kind":"listed","path":"scripts/generate_imdb_corpus.py","file_url":"https://github.com/dangitstam/topic-rnn/blob/HEAD/scripts/generate_imdb_corpus.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b156d9c122f71055"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}