{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bayesian-model-selection-the-marginal","title":"Bayesian Model Selection, the Marginal Likelihood, and Generalization","arxiv_id":"2202.11678","date":"2022-02-23","proceeding":null,"authors":["Sanae Lotfi","Pavel Izmailov","Gregory Benton","Micah Goldblum","Andrew Gordon Wilson"],"abstract":"How do we compare between hypotheses that are entirely consistent with observations? The marginal likelihood (aka Bayesian evidence), which represents the probability of generating our observations from a prior, provides a distinctive approach to this foundational question, automatically encoding Occam's razor. Although it has been observed that the marginal likelihood can overfit and is sensitive to prior assumptions, its limitations for hyperparameter learning and discrete model comparison have not been thoroughly investigated. We first revisit the appealing properties of the marginal likelihood for learning constraints and hypothesis testing. We then highlight the conceptual and practical issues in using the marginal likelihood as a proxy for generalization. Namely, we show how marginal likelihood can be negatively correlated with generalization, with implications for neural architecture search, and can lead to both underfitting and overfitting in hyperparameter learning. We also re-examine the connection between the marginal likelihood and PAC-Bayes bounds and use this connection to further elucidate the shortcomings of the marginal likelihood for model selection. We provide a partial remedy through a conditional marginal likelihood, which we show is more aligned with generalization, and practically valuable for large-scale hyperparameter learning, such as in deep kernel learning.","url_abs":"https://arxiv.org/abs/2202.11678v3","url_pdf":"https://arxiv.org/pdf/2202.11678v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bayesian-model-selection-the-marginal","repo_url":"https://github.com/sanaelotfi/bayesian_model_comparison","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"model-selection","task_name":"Model Selection"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"},{"task_slug":"model","task_name":"model"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2202.11678","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2202.11678"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sanaelotfi/bayesian_model_comparison","reach":null}],"summary":{"ran_draft_wrong":2,"ran_honours":1,"ran_fixture":2},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7ae3dc063aaedd3f","entry":"CondtionalMLL","repo":"sanaelotfi/bayesian_model_comparison","repo_kind":"official","path":"DKL_experiments/exact_runner.py","file_url":"https://github.com/sanaelotfi/bayesian_model_comparison/blob/HEAD/DKL_experiments/exact_runner.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7ae3dc063aaedd3f"}},{"code_sha256_prefix":"93fe84b4567ceb63","entry":"RMSE","repo":"sanaelotfi/bayesian_model_comparison","repo_kind":"official","path":"DKL_experiments/exact_runner.py","file_url":"https://github.com/sanaelotfi/bayesian_model_comparison/blob/HEAD/DKL_experiments/exact_runner.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"93fe84b4567ceb63"}},{"code_sha256_prefix":"5ea6bb4d9dff8b19","entry":"get_bma_acc","repo":"sanaelotfi/bayesian_model_comparison","repo_kind":"official","path":"Laplace_experiments/cifar/logcml_cifar100_cnns.py","file_url":"https://github.com/sanaelotfi/bayesian_model_comparison/blob/HEAD/Laplace_experiments/cifar/logcml_cifar100_cnns.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5ea6bb4d9dff8b19"}},{"code_sha256_prefix":"6f28b7641dcb5917","entry":"get_ll","repo":"sanaelotfi/bayesian_model_comparison","repo_kind":"official","path":"Laplace_experiments/cifar/logcml_cifar100_cnns.py","file_url":"https://github.com/sanaelotfi/bayesian_model_comparison/blob/HEAD/Laplace_experiments/cifar/logcml_cifar100_cnns.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6f28b7641dcb5917"}},{"code_sha256_prefix":"d7b8514511255eb4","entry":"get_mll","repo":"sanaelotfi/bayesian_model_comparison","repo_kind":"official","path":"DKL_experiments/exact_runner.py","file_url":"https://github.com/sanaelotfi/bayesian_model_comparison/blob/HEAD/DKL_experiments/exact_runner.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d7b8514511255eb4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}