{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/antisymmetricrnn-a-dynamical-system-view-on","title":"AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks","arxiv_id":"1902.09689","date":"2019-02-26","proceeding":"ICLR 2019 5","authors":["Bo Chang","Minmin Chen","Eldad Haber","Ed H. Chi"],"abstract":"Recurrent neural networks have gained widespread use in modeling sequential\ndata. Learning long-term dependencies using these models remains difficult\nthough, due to exploding or vanishing gradients. In this paper, we draw\nconnections between recurrent networks and ordinary differential equations. A\nspecial form of recurrent networks called the AntisymmetricRNN is proposed\nunder this theoretical framework, which is able to capture long-term\ndependencies thanks to the stability property of its underlying differential\nequation. Existing approaches to improving RNN trainability often incur\nsignificant computation overhead. In comparison, AntisymmetricRNN achieves the\nsame goal by design. We showcase the advantage of this new architecture through\nextensive simulations and experiments. AntisymmetricRNN exhibits much more\npredictable dynamics. It outperforms regular LSTM models on tasks requiring\nlong-term memory and matches the performance on tasks where short-term\ndependencies dominate despite being much simpler.","url_abs":"http://arxiv.org/abs/1902.09689v1","url_pdf":"http://arxiv.org/pdf/1902.09689v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"antisymmetricrnn-a-dynamical-system-view-on","repo_url":"https://github.com/KurochkinAlexey/AntisymmetricRNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"sequential-image-classification","task_name":"Sequential Image Classification"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/sequential-image-classification-on-noise","task":"Sequential Image Classification","dataset":"noise padded CIFAR-10","model":"AntisymmetricRNN w/ gating","rank_in_archive_order":6,"of":7,"metrics":{"% Test Accuracy":"54.7"},"uses_additional_data":false},{"leaderboard":"/sota/sequential-image-classification-on-noise","task":"Sequential Image Classification","dataset":"noise padded CIFAR-10","model":"LSTM","rank_in_archive_order":7,"of":7,"metrics":{"% Test Accuracy":"11.6"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1902.09689","atlas_url":"https://app.syntology.ai/?focus=1902.09689","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}