{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unitary-evolution-recurrent-neural-networks","title":"Unitary Evolution Recurrent Neural Networks","arxiv_id":"1511.06464","date":"2015-11-20","proceeding":null,"authors":["Martin Arjovsky","Amar Shah","Yoshua Bengio"],"abstract":"Recurrent neural networks (RNNs) are notoriously difficult to train. When the\neigenvalues of the hidden to hidden weight matrix deviate from absolute value\n1, optimization becomes difficult due to the well studied issue of vanishing\nand exploding gradients, especially when trying to learn long-term\ndependencies. To circumvent this problem, we propose a new architecture that\nlearns a unitary weight matrix, with eigenvalues of absolute value exactly 1.\nThe challenge we address is that of parametrizing unitary matrices in a way\nthat does not require expensive computations (such as eigendecomposition) after\neach weight update. We construct an expressive unitary weight matrix by\ncomposing several structured matrices that act as building blocks with\nparameters to be learned. Optimization with this parameterization becomes\nfeasible only when considering hidden states in the complex domain. We\ndemonstrate the potential of this architecture by achieving state of the art\nresults in several hard tasks involving very long-term dependencies.","url_abs":"http://arxiv.org/abs/1511.06464v4","url_pdf":"http://arxiv.org/pdf/1511.06464v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unitary-evolution-recurrent-neural-networks","repo_url":"https://github.com/rand0musername/urnn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"unitary-evolution-recurrent-neural-networks","repo_url":"https://github.com/v0lta/Complex-gated-recurrent-neural-networks","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"sequential-image-classification","task_name":"Sequential Image Classification"}],"methods":[{"method_slug":"rmsprop","method_name":"RMSProp"},{"method_slug":"unitary-rnn","method_name":"Unitary RNN"},{"method_slug":"modrelu","method_name":"modReLU"}],"datasets_introduced":[],"methods_introduced":[{"slug":"unitary-rnn","name":"Unitary RNN","full_name":"Unitary RNN"},{"slug":"modrelu","name":"modReLU","full_name":"modReLU"}],"results":[{"leaderboard":"/sota/sequential-image-classification-on-sequential","task":"Sequential Image Classification","dataset":"Sequential MNIST","model":"LSTM","rank_in_archive_order":26,"of":30,"metrics":{"Permuted Accuracy":"88%","Unpermuted Accuracy":"98.2%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1511.06464","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1511.06464"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/v0lta/Complex-gated-recurrent-neural-networks","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rand0musername/urnn","reach":null}],"summary":{"ran_violates":1},"by_repo_kind":{"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"539d7ad3db3eb359","entry":"baseline_cm","repo":"rand0musername/urnn","repo_kind":"listed","path":"evals.py","file_url":"https://github.com/rand0musername/urnn/blob/HEAD/evals.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"539d7ad3db3eb359"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}