{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/contextual-recurrent-neural-networks","title":"Contextual Recurrent Neural Networks","arxiv_id":"1902.03455","date":"2019-02-09","proceeding":null,"authors":["Sam Wenke","Jim Fleming"],"abstract":"There is an implicit assumption that by unfolding recurrent neural networks\n(RNN) in finite time, the misspecification of choosing a zero value for the\ninitial hidden state is mitigated by later time steps. This assumption has been\nshown to work in practice and alternative initialization may be suggested but\noften overlooked. In this paper, we propose a method of parameterizing the\ninitial hidden state of an RNN. The resulting architecture, referred to as a\nContextual RNN, can be trained end-to-end. The performance on an associative\nretrieval task is found to improve by conditioning the RNN initial hidden state\non contextual information from the input sequence. Furthermore, we propose a\nnovel method of conditionally generating sequences using the hidden state\nparameterization of Contextual RNN.","url_abs":"http://arxiv.org/abs/1902.03455v1","url_pdf":"http://arxiv.org/pdf/1902.03455v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"contextual-recurrent-neural-networks","repo_url":"https://github.com/fomorians/contextual_rnn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1902.03455","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}