{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/independently-recurrent-neural-network-indrnn","title":"Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN","arxiv_id":"1803.04831","date":"2018-03-13","proceeding":"CVPR 2018 6","authors":["Shuai Li","Wanqing Li","Chris Cook","Ce Zhu","Yanbo Gao"],"abstract":"Recurrent neural networks (RNNs) have been widely used for processing\nsequential data. However, RNNs are commonly difficult to train due to the\nwell-known gradient vanishing and exploding problems and hard to learn\nlong-term patterns. Long short-term memory (LSTM) and gated recurrent unit\n(GRU) were developed to address these problems, but the use of hyperbolic\ntangent and the sigmoid action functions results in gradient decay over layers.\nConsequently, construction of an efficiently trainable deep network is\nchallenging. In addition, all the neurons in an RNN layer are entangled\ntogether and their behaviour is hard to interpret. To address these problems, a\nnew type of RNN, referred to as independently recurrent neural network\n(IndRNN), is proposed in this paper, where neurons in the same layer are\nindependent of each other and they are connected across layers. We have shown\nthat an IndRNN can be easily regulated to prevent the gradient exploding and\nvanishing problems while allowing the network to learn long-term dependencies.\nMoreover, an IndRNN can work with non-saturated activation functions such as\nrelu (rectified linear unit) and be still trained robustly. Multiple IndRNNs\ncan be stacked to construct a network that is deeper than the existing RNNs.\nExperimental results have shown that the proposed IndRNN is able to process\nvery long sequences (over 5000 time steps), can be used to construct very deep\nnetworks (21 layers used in the experiment) and still be trained robustly.\nBetter performances have been achieved on various tasks by using IndRNNs\ncompared with the traditional RNN and LSTM. The code is available at\nhttps://github.com/Sunnydreamrain/IndRNN_Theano_Lasagne.","url_abs":"http://arxiv.org/abs/1803.04831v3","url_pdf":"http://arxiv.org/pdf/1803.04831v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"independently-recurrent-neural-network-indrnn","repo_url":"https://github.com/Sunnydreamrain/IndRNN_Theano_Lasagne","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"independently-recurrent-neural-network-indrnn","repo_url":"https://github.com/StefOe/indrnn-pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"independently-recurrent-neural-network-indrnn","repo_url":"https://github.com/Sunnydreamrain/IndRNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"independently-recurrent-neural-network-indrnn","repo_url":"https://github.com/Sunnydreamrain/IndRNN_pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"independently-recurrent-neural-network-indrnn","repo_url":"https://github.com/TobiasLee/Text-Classification","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"independently-recurrent-neural-network-indrnn","repo_url":"https://github.com/amcs1729/Predicting-cloud-CPU-usage-on-Azure-data","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"independently-recurrent-neural-network-indrnn","repo_url":"https://github.com/batzner/indrnn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"independently-recurrent-neural-network-indrnn","repo_url":"https://github.com/lmnt-com/haste","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"independently-recurrent-neural-network-indrnn","repo_url":"https://github.com/secretlyvogon/IndRNNTF","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"independently-recurrent-neural-network-indrnn","repo_url":"https://github.com/secretlyvogon/Neural-Network-Implementations","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"independently-recurrent-neural-network-indrnn","repo_url":"https://github.com/trevor-richardson/rnn_zoo","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"sequential-image-classification","task_name":"Sequential Image Classification"},{"task_slug":"skeleton-based-action-recognition","task_name":"Skeleton Based Action Recognition"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/language-modelling-on-penn-treebank-character","task":"Language Modelling","dataset":"Penn Treebank (Character Level)","model":"IndRNN","rank_in_archive_order":11,"of":20,"metrics":{"Bit per Character (BPC)":"1.19"},"uses_additional_data":false},{"leaderboard":"/sota/sequential-image-classification-on-sequential","task":"Sequential Image Classification","dataset":"Sequential MNIST","model":"IndRNN","rank_in_archive_order":21,"of":30,"metrics":{"Permuted Accuracy":"96%","Unpermuted Accuracy":"99%"},"uses_additional_data":false},{"leaderboard":"/sota/skeleton-based-action-recognition-on-ntu-rgbd","task":"Skeleton Based Action Recognition","dataset":"NTU RGB+D","model":"Ind-RNN","rank_in_archive_order":108,"of":135,"metrics":{"Accuracy (CS)":"81.8","Accuracy (CV)":"88.0"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1803.04831","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1803.04831"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/secretlyvogon/IndRNNTF","reach":{"status":"ok","spdx":"GPL-3.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/TobiasLee/Text-Classification","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/secretlyvogon/Neural-Network-Implementations","reach":{"status":"ok","spdx":"GPL-3.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/trevor-richardson/rnn_zoo","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Sunnydreamrain/IndRNN_pytorch","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/StefOe/indrnn-pytorch","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lmnt-com/haste","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Sunnydreamrain/IndRNN_Theano_Lasagne","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Sunnydreamrain/IndRNN","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/amcs1729/Predicting-cloud-CPU-usage-on-Azure-data","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/batzner/indrnn","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_fixture":1,"unverified":4},"by_repo_kind":{"listed":{"samples":5,"ran":1,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"353a35739d84ff7d","entry":"check_bounds","repo":"StefOe/indrnn-pytorch","repo_kind":"listed","path":"indrnn.py","file_url":"https://github.com/StefOe/indrnn-pytorch/blob/HEAD/indrnn.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"353a35739d84ff7d"}},{"code_sha256_prefix":"1f430a28206da9a4","entry":"LayerNormGRUScript","repo":"lmnt-com/haste","repo_kind":"listed","path":"frameworks/pytorch/layer_norm_gru.py","file_url":"https://github.com/lmnt-com/haste/blob/HEAD/frameworks/pytorch/layer_norm_gru.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1f430a28206da9a4"}},{"code_sha256_prefix":"4c0ac89dfdd49cb8","entry":"LayerNormIndRNNScript","repo":"lmnt-com/haste","repo_kind":"listed","path":"frameworks/pytorch/layer_norm_indrnn.py","file_url":"https://github.com/lmnt-com/haste/blob/HEAD/frameworks/pytorch/layer_norm_indrnn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4c0ac89dfdd49cb8"}},{"code_sha256_prefix":"05bc88c9379472a6","entry":"LayerNormLSTMScript","repo":"lmnt-com/haste","repo_kind":"listed","path":"frameworks/pytorch/layer_norm_lstm.py","file_url":"https://github.com/lmnt-com/haste/blob/HEAD/frameworks/pytorch/layer_norm_lstm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"05bc88c9379472a6"}},{"code_sha256_prefix":"f26710d97fca3491","entry":"extract","repo":"lmnt-com/haste","repo_kind":"listed","path":"benchmarks/report.py","file_url":"https://github.com/lmnt-com/haste/blob/HEAD/benchmarks/report.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f26710d97fca3491"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}