{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/recognizable-information-bottleneck","title":"Recognizable Information Bottleneck","arxiv_id":"2304.14618","date":"2023-04-28","proceeding":null,"authors":["Yilin Lyu","Xin Liu","Mingyang Song","Xinyue Wang","Yaxin Peng","Tieyong Zeng","Liping Jing"],"abstract":"Information Bottlenecks (IBs) learn representations that generalize to unseen data by information compression. However, existing IBs are practically unable to guarantee generalization in real-world scenarios due to the vacuous generalization bound. The recent PAC-Bayes IB uses information complexity instead of information compression to establish a connection with the mutual information generalization bound. However, it requires the computation of expensive second-order curvature, which hinders its practical application. In this paper, we establish the connection between the recognizability of representations and the recent functional conditional mutual information (f-CMI) generalization bound, which is significantly easier to estimate. On this basis we propose a Recognizable Information Bottleneck (RIB) which regularizes the recognizability of representations through a recognizability critic optimized by density ratio matching under the Bregman divergence. Extensive experiments on several commonly used datasets demonstrate the effectiveness of the proposed method in regularizing the model and estimating the generalization gap.","url_abs":"https://arxiv.org/abs/2304.14618v1","url_pdf":"https://arxiv.org/pdf/2304.14618v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"recognizable-information-bottleneck","repo_url":"https://github.com/lvyilin/recogib","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2304.14618","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.14618"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lvyilin/recogib","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/lvyilin/RecogIB","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":10},"by_repo_kind":{"official":{"samples":11,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5cbb7a66cbb3c912","entry":"Model","repo":"lvyilin/RecogIB","repo_kind":"official","path":"models/my_cnn.py","file_url":"https://github.com/lvyilin/RecogIB/blob/HEAD/models/my_cnn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5cbb7a66cbb3c912"}},{"code_sha256_prefix":"a15adaeab3030cc2","entry":"accuracy","repo":"lvyilin/recogib","repo_kind":"official","path":"utils.py","file_url":"https://github.com/lvyilin/recogib/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a15adaeab3030cc2"}},{"code_sha256_prefix":"54a12ecaea085254","entry":"get_activation","repo":"lvyilin/recogib","repo_kind":"official","path":"models/my_cnn.py","file_url":"https://github.com/lvyilin/recogib/blob/HEAD/models/my_cnn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"54a12ecaea085254"}},{"code_sha256_prefix":"e650c704b6b1ec98","entry":"get_dataset_class_number","repo":"lvyilin/recogib","repo_kind":"official","path":"dataset.py","file_url":"https://github.com/lvyilin/recogib/blob/HEAD/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e650c704b6b1ec98"}},{"code_sha256_prefix":"466b2e64e1eb8003","entry":"get_free_gpu","repo":"lvyilin/recogib","repo_kind":"official","path":"utils.py","file_url":"https://github.com/lvyilin/recogib/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"466b2e64e1eb8003"}},{"code_sha256_prefix":"93fdcab4a12efcd6","entry":"get_network","repo":"lvyilin/recogib","repo_kind":"official","path":"models/my_cnn.py","file_url":"https://github.com/lvyilin/recogib/blob/HEAD/models/my_cnn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"93fdcab4a12efcd6"}},{"code_sha256_prefix":"5e07bc057e0cc703","entry":"get_sequential","repo":"lvyilin/recogib","repo_kind":"official","path":"models/my_cnn.py","file_url":"https://github.com/lvyilin/recogib/blob/HEAD/models/my_cnn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5e07bc057e0cc703"}},{"code_sha256_prefix":"72f8e0a1e8ab3b02","entry":"get_trained_network","repo":"lvyilin/recogib","repo_kind":"official","path":"train_cri.py","file_url":"https://github.com/lvyilin/recogib/blob/HEAD/train_cri.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"72f8e0a1e8ab3b02"}},{"code_sha256_prefix":"cd6049a3e9b44f81","entry":"get_transform","repo":"lvyilin/recogib","repo_kind":"official","path":"dataset.py","file_url":"https://github.com/lvyilin/recogib/blob/HEAD/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd6049a3e9b44f81"}},{"code_sha256_prefix":"c82aa3d2032e5057","entry":"main_critic","repo":"lvyilin/recogib","repo_kind":"official","path":"train_cri.py","file_url":"https://github.com/lvyilin/recogib/blob/HEAD/train_cri.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c82aa3d2032e5057"}},{"code_sha256_prefix":"198885e8a8fa5d59","entry":"to_tensor_dataset","repo":"lvyilin/recogib","repo_kind":"official","path":"dataset.py","file_url":"https://github.com/lvyilin/recogib/blob/HEAD/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"198885e8a8fa5d59"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}