{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/can-recurrent-neural-networks-warp-time","title":"Can recurrent neural networks warp time?","arxiv_id":"1804.11188","date":"2018-03-23","proceeding":"ICLR 2018 1","authors":["Corentin Tallec","Yann Ollivier"],"abstract":"Successful recurrent models such as long short-term memories (LSTMs) and\ngated recurrent units (GRUs) use ad hoc gating mechanisms. Empirically these\nmodels have been found to improve the learning of medium to long term temporal\ndependencies and to help with vanishing gradient issues. We prove that\nlearnable gates in a recurrent model formally provide quasi- invariance to\ngeneral time transformations in the input data. We recover part of the LSTM\narchitecture from a simple axiomatic approach. This result leads to a new way\nof initializing gate biases in LSTMs and GRUs. Ex- perimentally, this new\nchrono initialization is shown to greatly improve learning of long term\ndependencies, with minimal implementation effort.","url_abs":"http://arxiv.org/abs/1804.11188v1","url_pdf":"http://arxiv.org/pdf/1804.11188v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"can-recurrent-neural-networks-warp-time","repo_url":"https://github.com/AravindGanesh/ChronoLSTM","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1804.11188","atlas_url":"https://app.syntology.ai/?focus=1804.11188","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}