{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ridge-regression-and-provable-deterministic-1","title":"Ridge Regression and Provable Deterministic Ridge Leverage Score Sampling","arxiv_id":"1803.06010","date":"2018-03-15","proceeding":"NeurIPS 2018","authors":["Shannon R. McCurdy"],"abstract":"Ridge leverage scores provide a balance between low-rank approximation and\nregularization, and are ubiquitous in randomized linear algebra and machine\nlearning. Deterministic algorithms are also of interest in the moderately big\ndata regime, because deterministic algorithms provide interpretability to the\npractitioner by having no failure probability and always returning the same\nresults.\n  We provide provable guarantees for deterministic column sampling using ridge\nleverage scores. The matrix sketch returned by our algorithm is a column subset\nof the original matrix, yielding additional interpretability. Like the\nrandomized counterparts, the deterministic algorithm provides (1 + {\\epsilon})\nerror column subset selection, (1 + {\\epsilon}) error projection-cost\npreservation, and an additive-multiplicative spectral bound. We also show that\nunder the assumption of power-law decay of ridge leverage scores, this\ndeterministic algorithm is provably as accurate as randomized algorithms.\n  Lastly, ridge regression is frequently used to regularize ill-posed linear\nleast-squares problems. While ridge regression provides shrinkage for the\nregression coefficients, many of the coefficients remain small but non-zero.\nPerforming ridge regression with the matrix sketch returned by our algorithm\nand a particular regularization parameter forces coefficients to zero and has a\nprovable (1 + {\\epsilon}) bound on the statistical risk. As such, it is an\ninteresting alternative to elastic net regularization.","url_abs":"http://arxiv.org/abs/1803.06010v2","url_pdf":"http://arxiv.org/pdf/1803.06010v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ridge-regression-and-provable-deterministic-1","repo_url":"https://github.com/srmcc/deterministic-ridge-leverage-sampling","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[{"task_slug":"regression-1","task_name":"regression"}],"methods":[{"method_slug":"interpretability","method_name":"Interpretability"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1803.06010","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}