{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lipschitz-recurrent-neural-networks","title":"Lipschitz Recurrent Neural Networks","arxiv_id":"2006.12070","date":"2020-06-22","proceeding":"ICLR 2021 1","authors":["N. Benjamin Erichson","Omri Azencot","Alejandro Queiruga","Liam Hodgkinson","Michael W. Mahoney"],"abstract":"Viewing recurrent neural networks (RNNs) as continuous-time dynamical systems, we propose a recurrent unit that describes the hidden state's evolution with two parts: a well-understood linear component plus a Lipschitz nonlinearity. This particular functional form facilitates stability analysis of the long-term behavior of the recurrent unit using tools from nonlinear systems theory. In turn, this enables architectural design decisions before experimentation. Sufficient conditions for global stability of the recurrent unit are obtained, motivating a novel scheme for constructing hidden-to-hidden matrices. Our experiments demonstrate that the Lipschitz RNN can outperform existing recurrent units on a range of benchmark tasks, including computer vision, language modeling and speech prediction tasks. Finally, through Hessian-based analysis we demonstrate that our Lipschitz recurrent unit is more robust with respect to input and parameter perturbations as compared to other continuous-time RNNs.","url_abs":"https://arxiv.org/abs/2006.12070v3","url_pdf":"https://arxiv.org/pdf/2006.12070v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lipschitz-recurrent-neural-networks","repo_url":"https://github.com/erichson/LipschitzRNN","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"sequential-image-classification","task_name":"Sequential Image Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/sequential-image-classification-on-sequential-1","task":"Sequential Image Classification","dataset":"Sequential CIFAR-10","model":"LipschitzRNN","rank_in_archive_order":10,"of":13,"metrics":{"Unpermuted Accuracy":"64.2"},"uses_additional_data":false},{"leaderboard":"/sota/sequential-image-classification-on-sequential","task":"Sequential Image Classification","dataset":"Sequential MNIST","model":"LipschitzRNN","rank_in_archive_order":19,"of":30,"metrics":{"Permuted Accuracy":"96.3%","Unpermuted Accuracy":"99.4"},"uses_additional_data":false},{"leaderboard":"/sota/sequential-image-classification-on-noise","task":"Sequential Image Classification","dataset":"noise padded CIFAR-10","model":"Lipschitz RNN","rank_in_archive_order":5,"of":7,"metrics":{"% Test Accuracy":"59.0"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2006.12070","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2006.12070"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/erichson/LipschitzRNN","reach":null}],"summary":{"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"a292b2b15cc48987","entry":"LipschitzRNN_ODE","repo":"erichson/LipschitzRNN","repo_kind":"official","path":"models.py","file_url":"https://github.com/erichson/LipschitzRNN/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"a292b2b15cc48987"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}