{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/compressing-neural-networks-using-the-1","title":"Compressing Neural Networks using the Variational Information Bottleneck","arxiv_id":"1802.10399","date":"2018-02-28","proceeding":"ICML 2018","authors":["Bin Dai","Chen Zhu","David Wipf"],"abstract":"Neural networks can be compressed to reduce memory and computational\nrequirements, or to increase accuracy by facilitating the use of a larger base\narchitecture. In this paper we focus on pruning individual neurons, which can\nsimultaneously trim model size, FLOPs, and run-time memory. To improve upon the\nperformance of existing compression algorithms we utilize the information\nbottleneck principle instantiated via a tractable variational bound.\nMinimization of this information theoretic bound reduces the redundancy between\nadjacent layers by aggregating useful information into a subset of neurons that\ncan be preserved. In contrast, the activations of disposable neurons are shut\noff via an attractive form of sparse regularization that emerges naturally from\nthis framework, providing tangible advantages over traditional sparsity\npenalties without contributing additional tuning parameters to the energy\nlandscape. We demonstrate state-of-the-art compression rates across an array of\ndatasets and network architectures.","url_abs":"http://arxiv.org/abs/1802.10399v3","url_pdf":"http://arxiv.org/pdf/1802.10399v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"compressing-neural-networks-using-the-1","repo_url":"https://github.com/zhuchen03/VIBNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1802.10399","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1802.10399"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhuchen03/VIBNet","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"listed":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f0c9a29156911331","entry":"accuracy","repo":"zhuchen03/VIBNet","repo_kind":"listed","path":"ib_vgg_train.py","file_url":"https://github.com/zhuchen03/VIBNet/blob/HEAD/ib_vgg_train.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f0c9a29156911331"}},{"code_sha256_prefix":"a744358ed458a3a2","entry":"reparameterize","repo":"zhuchen03/VIBNet","repo_kind":"listed","path":"ib_layers.py","file_url":"https://github.com/zhuchen03/VIBNet/blob/HEAD/ib_layers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a744358ed458a3a2"}},{"code_sha256_prefix":"56d9dbf9f393fbeb","entry":"validate","repo":"zhuchen03/VIBNet","repo_kind":"listed","path":"ib_vgg_train.py","file_url":"https://github.com/zhuchen03/VIBNet/blob/HEAD/ib_vgg_train.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"56d9dbf9f393fbeb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}