{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adaptive-network-sparsification-with","title":"Adaptive Network Sparsification with Dependent Variational Beta-Bernoulli Dropout","arxiv_id":"1805.10896","date":"2018-05-28","proceeding":null,"authors":["Juho Lee","Saehoon Kim","Jaehong Yoon","Hae Beom Lee","Eunho Yang","Sung Ju Hwang"],"abstract":"While variational dropout approaches have been shown to be effective for\nnetwork sparsification, they are still suboptimal in the sense that they set\nthe dropout rate for each neuron without consideration of the input data. With\nsuch input-independent dropout, each neuron is evolved to be generic across\ninputs, which makes it difficult to sparsify networks without accuracy loss. To\novercome this limitation, we propose adaptive variational dropout whose\nprobabilities are drawn from sparsity-inducing beta Bernoulli prior. It allows\neach neuron to be evolved either to be generic or specific for certain inputs,\nor dropped altogether. Such input-adaptive sparsity-inducing dropout allows the\nresulting network to tolerate larger degree of sparsity without losing its\nexpressive power by removing redundancies among features. We validate our\ndependent variational beta-Bernoulli dropout on multiple public datasets, on\nwhich it obtains significantly more compact networks than baseline methods,\nwith consistent accuracy improvements over the base networks.","url_abs":"http://arxiv.org/abs/1805.10896v3","url_pdf":"http://arxiv.org/pdf/1805.10896v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"adaptive-network-sparsification-with","repo_url":"https://github.com/OpenXAIProject/Variational_Dropouts","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[],"methods":[{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"variational-dropout","method_name":"Variational Dropout"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1805.10896","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1805.10896"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/OpenXAIProject/Variational_Dropouts","reach":null}],"summary":{"unverified":1},"by_repo_kind":{"listed":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"84cdf28af65b0915","entry":"ber_concrete","repo":"OpenXAIProject/Variational_Dropouts","repo_kind":"listed","path":"src/model/bbdropout.py","file_url":"https://github.com/OpenXAIProject/Variational_Dropouts/blob/HEAD/src/model/bbdropout.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"84cdf28af65b0915"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}