{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/optimal-errors-and-phase-transitions-in-high","title":"Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models","arxiv_id":"1708.03395","date":"2017-08-10","proceeding":null,"authors":["Jean Barbier","Florent Krzakala","Nicolas Macris","Léo Miolane","Lenka Zdeborová"],"abstract":"Generalized linear models (GLMs) arise in high-dimensional machine learning,\nstatistics, communications and signal processing. In this paper we analyze GLMs\nwhen the data matrix is random, as relevant in problems such as compressed\nsensing, error-correcting codes or benchmark models in neural networks. We\nevaluate the mutual information (or \"free entropy\") from which we deduce the\nBayes-optimal estimation and generalization errors. Our analysis applies to the\nhigh-dimensional limit where both the number of samples and the dimension are\nlarge and their ratio is fixed. Non-rigorous predictions for the optimal errors\nexisted for special cases of GLMs, e.g. for the perceptron, in the field of\nstatistical physics based on the so-called replica method. Our present paper\nrigorously establishes those decades old conjectures and brings forward their\nalgorithmic interpretation in terms of performance of the generalized\napproximate message-passing algorithm. Furthermore, we tightly characterize,\nfor many learning problems, regions of parameters for which this algorithm\nachieves the optimal performance, and locate the associated sharp phase\ntransitions separating learnable and non-learnable regions. We believe that\nthis random version of GLMs can serve as a challenging benchmark for\nmulti-purpose algorithms. This paper is divided in two parts that can be read\nindependently: The first part (main part) presents the model and main results,\ndiscusses some applications and sketches the main ideas of the proof. The\nsecond part (supplementary informations) is much more detailed and provides\nmore examples as well as all the proofs.","url_abs":"http://arxiv.org/abs/1708.03395v3","url_pdf":"http://arxiv.org/pdf/1708.03395v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"optimal-errors-and-phase-transitions-in-high","repo_url":"https://github.com/sphinxteam/GeneralizedLinearModel2017","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"high","task_name":"Vocal Bursts Intensity Prediction"},{"task_slug":"compressed-sensing","task_name":"compressed sensing"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1708.03395","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}