{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bayesian-compression-for-natural-language","title":"Bayesian Compression for Natural Language Processing","arxiv_id":"1810.10927","date":"2018-10-25","proceeding":"EMNLP 2018 10","authors":["Nadezhda Chirkova","Ekaterina Lobacheva","Dmitry Vetrov"],"abstract":"In natural language processing, a lot of the tasks are successfully solved\nwith recurrent neural networks, but such models have a huge number of\nparameters. The majority of these parameters are often concentrated in the\nembedding layer, which size grows proportionally to the vocabulary length. We\npropose a Bayesian sparsification technique for RNNs which allows compressing\nthe RNN dozens or hundreds of times without time-consuming hyperparameters\ntuning. We also generalize the model for vocabulary sparsification to filter\nout unnecessary words and compress the RNN even further. We show that the\nchoice of the kept words is interpretable. Code is available on github:\nhttps://github.com/tipt0p/SparseBayesianRNN","url_abs":"http://arxiv.org/abs/1810.10927v2","url_pdf":"http://arxiv.org/pdf/1810.10927v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bayesian-compression-for-natural-language","repo_url":"https://github.com/tipt0p/SparseBayesianRNN","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"bayesian-compression-for-natural-language","repo_url":"https://github.com/ars-ashuha/variational-dropout-sparsifies-dnn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"bayesian-compression-for-natural-language","repo_url":"https://github.com/senya-ashukha/variational-dropout-sparsifies-dnn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1810.10927","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}