{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/entropy-mcmc-sampling-from-flat-basins-with","title":"Entropy-MCMC: Sampling from Flat Basins with Ease","arxiv_id":"2310.05401","date":"2023-10-09","proceeding":null,"authors":["Bolian Li","Ruqi Zhang"],"abstract":"Bayesian deep learning counts on the quality of posterior distribution estimation. However, the posterior of deep neural networks is highly multi-modal in nature, with local modes exhibiting varying generalization performance. Given a practical budget, targeting at the original posterior can lead to suboptimal performance, as some samples may become trapped in \"bad\" modes and suffer from overfitting. Leveraging the observation that \"good\" modes with low generalization error often reside in flat basins of the energy landscape, we propose to bias sampling on the posterior toward these flat regions. Specifically, we introduce an auxiliary guiding variable, the stationary distribution of which resembles a smoothed posterior free from sharp modes, to lead the MCMC sampler to flat basins. By integrating this guiding variable with the model parameter, we create a simple joint distribution that enables efficient sampling with minimal computational overhead. We prove the convergence of our method and further show that it converges faster than several existing flatness-aware methods in the strongly convex setting. Empirical results demonstrate that our method can successfully sample from flat basins of the posterior, and outperforms all compared baselines on multiple benchmarks including classification, calibration, and out-of-distribution detection.","url_abs":"https://arxiv.org/abs/2310.05401v5","url_pdf":"https://arxiv.org/pdf/2310.05401v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"entropy-mcmc-sampling-from-flat-basins-with","repo_url":"https://github.com/lblaoke/emcmc","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"out-of-distribution-detection","task_name":"Out-of-Distribution Detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.05401","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.05401"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lblaoke/emcmc","reach":{"status":"ok"}}],"summary":{"ran_fixture":1,"ran":4,"unverified":3},"by_repo_kind":{"official":{"samples":8,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"131a82fd65128218","entry":"accuracy","repo":"lblaoke/emcmc","repo_kind":"official","path":"exp/imagenet_emcmc.py","file_url":"https://github.com/lblaoke/emcmc/blob/HEAD/exp/imagenet_emcmc.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"131a82fd65128218"}},{"code_sha256_prefix":"505bd5bddb925b70","entry":"cifar","repo":"lblaoke/emcmc","repo_kind":"official","path":"data_loader.py","file_url":"https://github.com/lblaoke/emcmc/blob/HEAD/data_loader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"505bd5bddb925b70"}},{"code_sha256_prefix":"2e669216626c55b3","entry":"noise","repo":"lblaoke/emcmc","repo_kind":"official","path":"exp/mnist_logistic_reg_emcmc.py","file_url":"https://github.com/lblaoke/emcmc/blob/HEAD/exp/mnist_logistic_reg_emcmc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2e669216626c55b3"}},{"code_sha256_prefix":"20f28b2642d8ffa2","entry":"noise","repo":"lblaoke/emcmc","repo_kind":"official","path":"exp/imagenet_sgld.py","file_url":"https://github.com/lblaoke/emcmc/blob/HEAD/exp/imagenet_sgld.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"20f28b2642d8ffa2"}},{"code_sha256_prefix":"7f3cbff84214f8f8","entry":"reg","repo":"lblaoke/emcmc","repo_kind":"official","path":"exp/mnist_logistic_reg_emcmc.py","file_url":"https://github.com/lblaoke/emcmc/blob/HEAD/exp/mnist_logistic_reg_emcmc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7f3cbff84214f8f8"}},{"code_sha256_prefix":"f97bee323b69233f","entry":"imagenet","repo":"lblaoke/emcmc","repo_kind":"official","path":"data_loader.py","file_url":"https://github.com/lblaoke/emcmc/blob/HEAD/data_loader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f97bee323b69233f"}},{"code_sha256_prefix":"76887d631d72b05f","entry":"score","repo":"lblaoke/emcmc","repo_kind":"official","path":"metric.py","file_url":"https://github.com/lblaoke/emcmc/blob/HEAD/metric.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"76887d631d72b05f"}},{"code_sha256_prefix":"54b7a21c70e8d57f","entry":"svhn","repo":"lblaoke/emcmc","repo_kind":"official","path":"data_loader.py","file_url":"https://github.com/lblaoke/emcmc/blob/HEAD/data_loader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"54b7a21c70e8d57f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}