{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/phased-lstm-accelerating-recurrent-network","title":"Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences","arxiv_id":"1610.09513","date":"2016-10-29","proceeding":"NeurIPS 2016 12","authors":["Daniel Neil","Michael Pfeiffer","Shih-Chii Liu"],"abstract":"Recurrent Neural Networks (RNNs) have become the state-of-the-art choice for\nextracting patterns from temporal sequences. However, current RNN models are\nill-suited to process irregularly sampled data triggered by events generated in\ncontinuous time by sensors or other neurons. Such data can occur, for example,\nwhen the input comes from novel event-driven artificial sensors that generate\nsparse, asynchronous streams of events or from multiple conventional sensors\nwith different update intervals. In this work, we introduce the Phased LSTM\nmodel, which extends the LSTM unit by adding a new time gate. This gate is\ncontrolled by a parametrized oscillation with a frequency range that produces\nupdates of the memory cell only during a small percentage of the cycle. Even\nwith the sparse updates imposed by the oscillation, the Phased LSTM network\nachieves faster convergence than regular LSTMs on tasks which require learning\nof long sequences. The model naturally integrates inputs from sensors of\narbitrary sampling rates, thereby opening new areas of investigation for\nprocessing asynchronous sensory events that carry timing information. It also\ngreatly improves the performance of LSTMs in standard RNN applications, and\ndoes so with an order-of-magnitude fewer computes at runtime.","url_abs":"http://arxiv.org/abs/1610.09513v1","url_pdf":"http://arxiv.org/pdf/1610.09513v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"phased-lstm-accelerating-recurrent-network","repo_url":"https://github.com/DarryO/time_lstm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"phased-lstm-accelerating-recurrent-network","repo_url":"https://github.com/fferroni/PhasedLSTM-Keras","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"phased-lstm-accelerating-recurrent-network","repo_url":"https://github.com/muhdhuz/MTCRNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"phased-lstm-accelerating-recurrent-network","repo_url":"https://github.com/philipperemy/tensorflow-phased-lstm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"temporal-sequences","task_name":"Temporal Sequences"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1610.09513","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}