{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-variational-inference-for-sparse-1","title":"Efficient Variational Inference for Sparse Deep Learning with Theoretical Guarantee","arxiv_id":"2011.07439","date":"2020-11-15","proceeding":"NeurIPS 2020 12","authors":["Jincheng Bai","Qifan Song","Guang Cheng"],"abstract":"Sparse deep learning aims to address the challenge of huge storage consumption by deep neural networks, and to recover the sparse structure of target functions. Although tremendous empirical successes have been achieved, most sparse deep learning algorithms are lacking of theoretical support. On the other hand, another line of works have proposed theoretical frameworks that are computationally infeasible. In this paper, we train sparse deep neural networks with a fully Bayesian treatment under spike-and-slab priors, and develop a set of computationally efficient variational inferences via continuous relaxation of Bernoulli distribution. The variational posterior contraction rate is provided, which justifies the consistency of the proposed variational Bayes method. Notably, our empirical results demonstrate that this variational procedure provides uncertainty quantification in terms of Bayesian predictive distribution and is also capable to accomplish consistent variable selection by training a sparse multi-layer neural network.","url_abs":"https://arxiv.org/abs/2011.07439v1","url_pdf":"https://arxiv.org/pdf/2011.07439v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-variational-inference-for-sparse-1","repo_url":"https://github.com/JinchengBai/sparse-variational-bnn","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"uncertainty-quantification","task_name":"Uncertainty Quantification"},{"task_slug":"variable-selection","task_name":"Variable Selection"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2011.07439","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2011.07439"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/JinchengBai/sparse-variational-bnn","reach":null}],"summary":{"ran":2,"ran_draft_wrong":3,"ran_honours":1},"by_repo_kind":{"official":{"samples":6,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"b68271e56ba25185","entry":"MLPLayer","repo":"JinchengBai/sparse-variational-bnn","repo_kind":"official","path":"models/sparsefunc.py","file_url":"https://github.com/JinchengBai/sparse-variational-bnn/blob/HEAD/models/sparsefunc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b68271e56ba25185"}},{"code_sha256_prefix":"4f8c1da3635a080d","entry":"SFunc","repo":"JinchengBai/sparse-variational-bnn","repo_kind":"official","path":"models/sparsefunc.py","file_url":"https://github.com/JinchengBai/sparse-variational-bnn/blob/HEAD/models/sparsefunc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4f8c1da3635a080d"}},{"code_sha256_prefix":"ef0219182bdf035d","entry":"gumbel_softmax","repo":"JinchengBai/sparse-variational-bnn","repo_kind":"official","path":"models/sparsefunc.py","file_url":"https://github.com/JinchengBai/sparse-variational-bnn/blob/HEAD/models/sparsefunc.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ef0219182bdf035d"}},{"code_sha256_prefix":"744b361b7cfdf271","entry":"log_gaussian","repo":"JinchengBai/sparse-variational-bnn","repo_kind":"official","path":"models/sparsefunc.py","file_url":"https://github.com/JinchengBai/sparse-variational-bnn/blob/HEAD/models/sparsefunc.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"744b361b7cfdf271"}},{"code_sha256_prefix":"0126ad8ebd601b60","entry":"logit","repo":"JinchengBai/sparse-variational-bnn","repo_kind":"official","path":"models/sparsefunc.py","file_url":"https://github.com/JinchengBai/sparse-variational-bnn/blob/HEAD/models/sparsefunc.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0126ad8ebd601b60"}},{"code_sha256_prefix":"6ed27939405d77df","entry":"sigmoid","repo":"JinchengBai/sparse-variational-bnn","repo_kind":"official","path":"models/sparsefunc.py","file_url":"https://github.com/JinchengBai/sparse-variational-bnn/blob/HEAD/models/sparsefunc.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6ed27939405d77df"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}