{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/maximizing-the-information-learned-from","title":"Maximizing the information learned from finite data selects a simple model","arxiv_id":"1705.01166","date":"2017-05-02","proceeding":null,"authors":["Henry H. Mattingly","Mark K. Transtrum","Michael C. Abbott","Benjamin B. Machta"],"abstract":"We use the language of uninformative Bayesian prior choice to study the\nselection of appropriately simple effective models. We advocate for the prior\nwhich maximizes the mutual information between parameters and predictions,\nlearning as much as possible from limited data. When many parameters are poorly\nconstrained by the available data, we find that this prior puts weight only on\nboundaries of the parameter manifold. Thus it selects a lower-dimensional\neffective theory in a principled way, ignoring irrelevant parameter directions.\nIn the limit where there is sufficient data to tightly constrain any number of\nparameters, this reduces to Jeffreys prior. But we argue that this limit is\npathological when applied to the hyper-ribbon parameter manifolds generic in\nscience, because it leads to dramatic dependence on effects invisible to\nexperiment.","url_abs":"http://arxiv.org/abs/1705.01166v3","url_pdf":"http://arxiv.org/pdf/1705.01166v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"maximizing-the-information-learned-from","repo_url":"https://github.com/mcabbott/RationalIgnorance.jl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"maximizing-the-information-learned-from","repo_url":"https://github.com/mcabbott/atomicpriors.jl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}