{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unicornn-a-recurrent-model-for-learning-very","title":"UnICORNN: A recurrent model for learning very long time dependencies","arxiv_id":"2103.05487","date":"2021-03-09","proceeding":null,"authors":["T. Konstantin Rusch","Siddhartha Mishra"],"abstract":"The design of recurrent neural networks (RNNs) to accurately process sequential inputs with long-time dependencies is very challenging on account of the exploding and vanishing gradient problem. To overcome this, we propose a novel RNN architecture which is based on a structure preserving discretization of a Hamiltonian system of second-order ordinary differential equations that models networks of oscillators. The resulting RNN is fast, invertible (in time), memory efficient and we derive rigorous bounds on the hidden state gradients to prove the mitigation of the exploding and vanishing gradient problem. A suite of experiments are presented to demonstrate that the proposed RNN provides state of the art performance on a variety of learning tasks with (very) long-time dependencies.","url_abs":"https://arxiv.org/abs/2103.05487v2","url_pdf":"https://arxiv.org/pdf/2103.05487v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unicornn-a-recurrent-model-for-learning-very","repo_url":"https://github.com/tk-rusch/unicornn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"sequential-image-classification","task_name":"Sequential Image Classification"},{"task_slug":"time-series","task_name":"Time Series Analysis"},{"task_slug":"time-series-classification","task_name":"Time Series Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/sentiment-analysis-on-imdb","task":"Sentiment Analysis","dataset":"IMDb","model":"UnICORNN","rank_in_archive_order":39,"of":49,"metrics":{"Accuracy":"88.4"},"uses_additional_data":false},{"leaderboard":"/sota/sequential-image-classification-on-sequential","task":"Sequential Image Classification","dataset":"Sequential MNIST","model":"UnICORNN","rank_in_archive_order":7,"of":30,"metrics":{"Permuted Accuracy":"98.4"},"uses_additional_data":false},{"leaderboard":"/sota/sequential-image-classification-on-noise","task":"Sequential Image Classification","dataset":"noise padded CIFAR-10","model":"UnICORNN","rank_in_archive_order":2,"of":7,"metrics":{"% Test Accuracy":"62.4"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-classification-on-eigenworms","task":"Time Series Classification","dataset":"EigenWorms","model":"UnICORNN","rank_in_archive_order":2,"of":8,"metrics":{"% Test Accuracy":"90.3"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-classification-on-eigenworms","task":"Time Series Classification","dataset":"EigenWorms","model":"coRNN","rank_in_archive_order":3,"of":8,"metrics":{"% Test Accuracy":"86.7"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-classification-on-eigenworms","task":"Time Series Classification","dataset":"EigenWorms","model":"IndRNN","rank_in_archive_order":6,"of":8,"metrics":{"% Test Accuracy":"49.7"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-classification-on-eigenworms","task":"Time Series Classification","dataset":"EigenWorms","model":"expRNN","rank_in_archive_order":8,"of":8,"metrics":{"% Test Accuracy":"40.0"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2103.05487","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.05487"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tk-rusch/unicornn","reach":null}],"summary":{"ran_honours":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"fd308c50fdd7ff27","entry":"binary_accuracy","repo":"tk-rusch/unicornn","repo_kind":"official","path":"IMDB/IMDB_task.py","file_url":"https://github.com/tk-rusch/unicornn/blob/HEAD/IMDB/IMDB_task.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fd308c50fdd7ff27"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}