{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bmrs-bayesian-model-reduction-for-structured","title":"BMRS: Bayesian Model Reduction for Structured Pruning","arxiv_id":"2406.01345","date":"2024-06-03","proceeding":null,"authors":["Dustin Wright","Christian Igel","Raghavendra Selvan"],"abstract":"Modern neural networks are often massively overparameterized leading to high compute costs during training and at inference. One effective method to improve both the compute and energy efficiency of neural networks while maintaining good performance is structured pruning, where full network structures (e.g.~neurons or convolutional filters) that have limited impact on the model output are removed. In this work, we propose Bayesian Model Reduction for Structured pruning (BMRS), a fully end-to-end Bayesian method of structured pruning. BMRS is based on two recent methods: Bayesian structured pruning with multiplicative noise, and Bayesian model reduction (BMR), a method which allows efficient comparison of Bayesian models under a change in prior. We present two realizations of BMRS derived from different priors which yield different structured pruning characteristics: 1) BMRS_N with the truncated log-normal prior, which offers reliable compression rates and accuracy without the need for tuning any thresholds and 2) BMRS_U with the truncated log-uniform prior that can achieve more aggressive compression based on the boundaries of truncation. Overall, we find that BMRS offers a theoretically grounded approach to structured pruning of neural networks yielding both high compression rates and accuracy. Experiments on multiple datasets and neural networks of varying complexity showed that the two BMRS methods offer a competitive performance-efficiency trade-off compared to other pruning methods.","url_abs":"https://arxiv.org/abs/2406.01345v2","url_pdf":"https://arxiv.org/pdf/2406.01345v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bmrs-bayesian-model-reduction-for-structured","repo_url":"https://github.com/saintslab/bmrs-structured-pruning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.01345","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.01345"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/saintslab/bmrs-structured-pruning","reach":null}],"summary":{"ran_honours":9,"ran":1,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":11,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"b4fac9801138a0cd","entry":"Etheta","repo":"saintslab/bmrs-structured-pruning","repo_kind":"official","path":"modules/layers.py","file_url":"https://github.com/saintslab/bmrs-structured-pruning/blob/HEAD/modules/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b4fac9801138a0cd"}},{"code_sha256_prefix":"6acafad15c4789d3","entry":"LogUniformApproximateDiracBMRPruningLayer","repo":"saintslab/bmrs-structured-pruning","repo_kind":"official","path":"modules/layers.py","file_url":"https://github.com/saintslab/bmrs-structured-pruning/blob/HEAD/modules/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6acafad15c4789d3"}},{"code_sha256_prefix":"4b2929f719750d8a","entry":"Phi","repo":"saintslab/bmrs-structured-pruning","repo_kind":"official","path":"modules/layers.py","file_url":"https://github.com/saintslab/bmrs-structured-pruning/blob/HEAD/modules/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4b2929f719750d8a"}},{"code_sha256_prefix":"1578c441ee66a2c3","entry":"Phi_inv","repo":"saintslab/bmrs-structured-pruning","repo_kind":"official","path":"modules/layers.py","file_url":"https://github.com/saintslab/bmrs-structured-pruning/blob/HEAD/modules/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1578c441ee66a2c3"}},{"code_sha256_prefix":"07d6377d231bd408","entry":"calculate_model_kl","repo":"saintslab/bmrs-structured-pruning","repo_kind":"official","path":"modules/layers.py","file_url":"https://github.com/saintslab/bmrs-structured-pruning/blob/HEAD/modules/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"07d6377d231bd408"}},{"code_sha256_prefix":"e4ccef0f9ddf163c","entry":"erfcx","repo":"saintslab/bmrs-structured-pruning","repo_kind":"official","path":"modules/layers.py","file_url":"https://github.com/saintslab/bmrs-structured-pruning/blob/HEAD/modules/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e4ccef0f9ddf163c"}},{"code_sha256_prefix":"6478df241893f8ce","entry":"gather_pruning_layers","repo":"saintslab/bmrs-structured-pruning","repo_kind":"official","path":"modules/layers.py","file_url":"https://github.com/saintslab/bmrs-structured-pruning/blob/HEAD/modules/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6478df241893f8ce"}},{"code_sha256_prefix":"a62d90c72e6633f8","entry":"get_weight_l2","repo":"saintslab/bmrs-structured-pruning","repo_kind":"official","path":"modules/layers.py","file_url":"https://github.com/saintslab/bmrs-structured-pruning/blob/HEAD/modules/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a62d90c72e6633f8"}},{"code_sha256_prefix":"fbe81395901bb87a","entry":"kldiv","repo":"saintslab/bmrs-structured-pruning","repo_kind":"official","path":"modules/layers.py","file_url":"https://github.com/saintslab/bmrs-structured-pruning/blob/HEAD/modules/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fbe81395901bb87a"}},{"code_sha256_prefix":"3f5012a9ea5a467a","entry":"phi","repo":"saintslab/bmrs-structured-pruning","repo_kind":"official","path":"modules/layers.py","file_url":"https://github.com/saintslab/bmrs-structured-pruning/blob/HEAD/modules/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3f5012a9ea5a467a"}},{"code_sha256_prefix":"d71a61ef4305cdc5","entry":"reparameterize","repo":"saintslab/bmrs-structured-pruning","repo_kind":"official","path":"modules/layers.py","file_url":"https://github.com/saintslab/bmrs-structured-pruning/blob/HEAD/modules/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d71a61ef4305cdc5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}