{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/exploring-sparsity-in-recurrent-neural","title":"Exploring Sparsity in Recurrent Neural Networks","arxiv_id":"1704.05119","date":"2017-04-17","proceeding":null,"authors":["Sharan Narang","Erich Elsen","Gregory Diamos","Shubho Sengupta"],"abstract":"Recurrent Neural Networks (RNN) are widely used to solve a variety of\nproblems and as the quantity of data and the amount of available compute have\nincreased, so have model sizes. The number of parameters in recent\nstate-of-the-art networks makes them hard to deploy, especially on mobile\nphones and embedded devices. The challenge is due to both the size of the model\nand the time it takes to evaluate it. In order to deploy these RNNs\nefficiently, we propose a technique to reduce the parameters of a network by\npruning weights during the initial training of the network. At the end of\ntraining, the parameters of the network are sparse while accuracy is still\nclose to the original dense neural network. The network size is reduced by 8x\nand the time required to train the model remains constant. Additionally, we can\nprune a larger dense network to achieve better than baseline performance while\nstill reducing the total number of parameters significantly. Pruning RNNs\nreduces the size of the model and can also help achieve significant inference\ntime speed-up using sparse matrix multiply. Benchmarks show that using our\ntechnique model size can be reduced by 90% and speed-up is around 2x to 7x.","url_abs":"http://arxiv.org/abs/1704.05119v2","url_pdf":"http://arxiv.org/pdf/1704.05119v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"exploring-sparsity-in-recurrent-neural","repo_url":"https://github.com/puhsu/pruning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1704.05119","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1704.05119"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/puhsu/pruning","reach":null}],"summary":{"ran_honours":1},"by_repo_kind":{"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"6aaf07ccda31ac5f","entry":"repackage_hidden","repo":"puhsu/pruning","repo_kind":"listed","path":"model.py","file_url":"https://github.com/puhsu/pruning/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6aaf07ccda31ac5f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}