{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bayesian-sparsification-of-gated-recurrent","title":"Bayesian Sparsification of Gated Recurrent Neural Networks","arxiv_id":"1812.05692","date":"2018-12-12","proceeding":"NIPS Workshop CDNNRIA 2018","authors":["Ekaterina Lobacheva","Nadezhda Chirkova","Dmitry Vetrov"],"abstract":"Bayesian methods have been successfully applied to sparsify weights of neural\nnetworks and to remove structure units from the networks, e. g. neurons. We\napply and further develop this approach for gated recurrent architectures.\nSpecifically, in addition to sparsification of individual weights and neurons,\nwe propose to sparsify preactivations of gates and information flow in LSTM. It\nmakes some gates and information flow components constant, speeds up forward\npass and improves compression. Moreover, the resulting structure of gate\nsparsity is interpretable and depends on the task. Code is available on github:\nhttps://github.com/tipt0p/SparseBayesianRNN","url_abs":"http://arxiv.org/abs/1812.05692v1","url_pdf":"http://arxiv.org/pdf/1812.05692v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bayesian-sparsification-of-gated-recurrent","repo_url":"https://github.com/tipt0p/SparseBayesianRNN","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}