{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-compact-representation-for-bayesian-neural","title":"A Compact Representation for Bayesian Neural Networks By Removing Permutation Symmetry","arxiv_id":"2401.00611","date":"2023-12-31","proceeding":null,"authors":["Tim Z. Xiao","Weiyang Liu","Robert Bamler"],"abstract":"Bayesian neural networks (BNNs) are a principled approach to modeling predictive uncertainties in deep learning, which are important in safety-critical applications. Since exact Bayesian inference over the weights in a BNN is intractable, various approximate inference methods exist, among which sampling methods such as Hamiltonian Monte Carlo (HMC) are often considered the gold standard. While HMC provides high-quality samples, it lacks interpretable summary statistics because its sample mean and variance is meaningless in neural networks due to permutation symmetry. In this paper, we first show that the role of permutations can be meaningfully quantified by a number of transpositions metric. We then show that the recently proposed rebasin method allows us to summarize HMC samples into a compact representation that provides a meaningful explicit uncertainty estimate for each weight in a neural network, thus unifying sampling methods with variational inference. We show that this compact representation allows us to compare trained BNNs directly in weight space across sampling methods and variational inference, and to efficiently prune neural networks trained without explicit Bayesian frameworks by exploiting uncertainty estimates from HMC.","url_abs":"https://arxiv.org/abs/2401.00611v1","url_pdf":"https://arxiv.org/pdf/2401.00611v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-compact-representation-for-bayesian-neural","repo_url":"https://github.com/timxzz/abi_with_rebasin","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"bayesian-inference","task_name":"Bayesian Inference"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2401.00611","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.00611"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/timxzz/abi_with_rebasin","reach":{"status":"ok"}}],"summary":{"ran":9,"unverified":5},"by_repo_kind":{"official":{"samples":14,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"10b8964e0c906ead","entry":"agreement","repo":"timxzz/abi_with_rebasin","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/timxzz/abi_with_rebasin/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"10b8964e0c906ead"}},{"code_sha256_prefix":"6da2c42649dcb1bb","entry":"construct_config_string","repo":"timxzz/abi_with_rebasin","repo_kind":"official","path":"MNIST_HMC.py","file_url":"https://github.com/timxzz/abi_with_rebasin/blob/HEAD/MNIST_HMC.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6da2c42649dcb1bb"}},{"code_sha256_prefix":"fe0567580758cd02","entry":"cost_matrix","repo":"timxzz/abi_with_rebasin","repo_kind":"official","path":"rebasin.py","file_url":"https://github.com/timxzz/abi_with_rebasin/blob/HEAD/rebasin.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fe0567580758cd02"}},{"code_sha256_prefix":"b74ad0eadfe95127","entry":"load_hmc_samples","repo":"timxzz/abi_with_rebasin","repo_kind":"official","path":"utils.py","file_url":"https://github.com/timxzz/abi_with_rebasin/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b74ad0eadfe95127"}},{"code_sha256_prefix":"3784325e58d559a6","entry":"object_to_flatten_dict","repo":"timxzz/abi_with_rebasin","repo_kind":"official","path":"MNIST_HMC.py","file_url":"https://github.com/timxzz/abi_with_rebasin/blob/HEAD/MNIST_HMC.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3784325e58d559a6"}},{"code_sha256_prefix":"2efb1c739594d5ee","entry":"to_numpy","repo":"timxzz/abi_with_rebasin","repo_kind":"official","path":"MNIST_models.py","file_url":"https://github.com/timxzz/abi_with_rebasin/blob/HEAD/MNIST_models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2efb1c739594d5ee"}},{"code_sha256_prefix":"5567788db47b69bc","entry":"total_variation_distance","repo":"timxzz/abi_with_rebasin","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/timxzz/abi_with_rebasin/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5567788db47b69bc"}},{"code_sha256_prefix":"a930a033d7a52021","entry":"train_bnn","repo":"timxzz/abi_with_rebasin","repo_kind":"official","path":"MNIST_VI.py","file_url":"https://github.com/timxzz/abi_with_rebasin/blob/HEAD/MNIST_VI.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a930a033d7a52021"}},{"code_sha256_prefix":"570cf6f632bc4a19","entry":"w2_distance","repo":"timxzz/abi_with_rebasin","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/timxzz/abi_with_rebasin/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"570cf6f632bc4a19"}},{"code_sha256_prefix":"be95ccb8d35d9e3a","entry":"acts_cost_matrix","repo":"timxzz/abi_with_rebasin","repo_kind":"official","path":"rebasin.py","file_url":"https://github.com/timxzz/abi_with_rebasin/blob/HEAD/rebasin.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"be95ccb8d35d9e3a"}},{"code_sha256_prefix":"0c008d33347d27ce","entry":"closest_permutation_by_activation","repo":"timxzz/abi_with_rebasin","repo_kind":"official","path":"rebasin.py","file_url":"https://github.com/timxzz/abi_with_rebasin/blob/HEAD/rebasin.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0c008d33347d27ce"}},{"code_sha256_prefix":"1e4d817efd08ff72","entry":"load_hmc_hyperparameters","repo":"timxzz/abi_with_rebasin","repo_kind":"official","path":"MNIST_HMC.py","file_url":"https://github.com/timxzz/abi_with_rebasin/blob/HEAD/MNIST_HMC.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1e4d817efd08ff72"}},{"code_sha256_prefix":"ec9c68eaf082f7f3","entry":"train","repo":"timxzz/abi_with_rebasin","repo_kind":"official","path":"MNIST_net_matching.py","file_url":"https://github.com/timxzz/abi_with_rebasin/blob/HEAD/MNIST_net_matching.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ec9c68eaf082f7f3"}},{"code_sha256_prefix":"3ac55ecfd1624ca5","entry":"train_bnn_step","repo":"timxzz/abi_with_rebasin","repo_kind":"official","path":"MNIST_VI.py","file_url":"https://github.com/timxzz/abi_with_rebasin/blob/HEAD/MNIST_VI.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3ac55ecfd1624ca5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}