{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dilated-recurrent-neural-networks","title":"Dilated Recurrent Neural Networks","arxiv_id":"1710.02224","date":"2017-10-05","proceeding":"NeurIPS 2017 12","authors":["Shiyu Chang","Yang Zhang","Wei Han","Mo Yu","Xiaoxiao Guo","Wei Tan","Xiaodong Cui","Michael Witbrock","Mark Hasegawa-Johnson","Thomas S. Huang"],"abstract":"Learning with recurrent neural networks (RNNs) on long sequences is a\nnotoriously difficult task. There are three major challenges: 1) complex\ndependencies, 2) vanishing and exploding gradients, and 3) efficient\nparallelization. In this paper, we introduce a simple yet effective RNN\nconnection structure, the DilatedRNN, which simultaneously tackles all of these\nchallenges. The proposed architecture is characterized by multi-resolution\ndilated recurrent skip connections and can be combined flexibly with diverse\nRNN cells. Moreover, the DilatedRNN reduces the number of parameters needed and\nenhances training efficiency significantly, while matching state-of-the-art\nperformance (even with standard RNN cells) in tasks involving very long-term\ndependencies. To provide a theory-based quantification of the architecture's\nadvantages, we introduce a memory capacity measure, the mean recurrent length,\nwhich is more suitable for RNNs with long skip connections than existing\nmeasures. We rigorously prove the advantages of the DilatedRNN over other\nrecurrent neural architectures. The code for our method is publicly available\nat https://github.com/code-terminator/DilatedRNN","url_abs":"http://arxiv.org/abs/1710.02224v3","url_pdf":"http://arxiv.org/pdf/1710.02224v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dilated-recurrent-neural-networks","repo_url":"https://github.com/code-terminator/DilatedRNN","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"dilated-recurrent-neural-networks","repo_url":"https://github.com/zalandoresearch/pytorch-dilated-rnn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"sequential-image-classification","task_name":"Sequential Image Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/sequential-image-classification-on-sequential","task":"Sequential Image Classification","dataset":"Sequential MNIST","model":"Dilated GRU","rank_in_archive_order":24,"of":30,"metrics":{"Permuted Accuracy":"94.6%","Unpermuted Accuracy":"99.2%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1710.02224","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1710.02224"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zalandoresearch/pytorch-dilated-rnn","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/code-terminator/DilatedRNN","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":6},"by_repo_kind":{"listed":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"70277a5008135fcc","entry":"batchify","repo":"zalandoresearch/pytorch-dilated-rnn","repo_kind":"listed","path":"lm.py","file_url":"https://github.com/zalandoresearch/pytorch-dilated-rnn/blob/HEAD/lm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"70277a5008135fcc"}},{"code_sha256_prefix":"b1a3998b3a88fc31","entry":"char_tensor","repo":"zalandoresearch/pytorch-dilated-rnn","repo_kind":"listed","path":"char_rnn/utils.py","file_url":"https://github.com/zalandoresearch/pytorch-dilated-rnn/blob/HEAD/char_rnn/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1a3998b3a88fc31"}},{"code_sha256_prefix":"f9eab345dc306e54","entry":"data_generator","repo":"zalandoresearch/pytorch-dilated-rnn","repo_kind":"listed","path":"copy_memory/utils.py","file_url":"https://github.com/zalandoresearch/pytorch-dilated-rnn/blob/HEAD/copy_memory/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9eab345dc306e54"}},{"code_sha256_prefix":"e33987634b6a19bf","entry":"get_batch","repo":"zalandoresearch/pytorch-dilated-rnn","repo_kind":"listed","path":"lm.py","file_url":"https://github.com/zalandoresearch/pytorch-dilated-rnn/blob/HEAD/lm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e33987634b6a19bf"}},{"code_sha256_prefix":"b0edd931ca8776ea","entry":"read_file","repo":"zalandoresearch/pytorch-dilated-rnn","repo_kind":"listed","path":"char_rnn/utils.py","file_url":"https://github.com/zalandoresearch/pytorch-dilated-rnn/blob/HEAD/char_rnn/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b0edd931ca8776ea"}},{"code_sha256_prefix":"5b50d8b3f7aa6aa7","entry":"repackage_hidden","repo":"zalandoresearch/pytorch-dilated-rnn","repo_kind":"listed","path":"lm.py","file_url":"https://github.com/zalandoresearch/pytorch-dilated-rnn/blob/HEAD/lm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5b50d8b3f7aa6aa7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}