{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/additive-approximations-in-high-dimensional","title":"Additive Approximations in High Dimensional Nonparametric Regression via the SALSA","arxiv_id":"1602.00287","date":"2016-01-31","proceeding":null,"authors":["Kirthevasan Kandasamy","Yao-Liang Yu"],"abstract":"High dimensional nonparametric regression is an inherently difficult problem\nwith known lower bounds depending exponentially in dimension. A popular\nstrategy to alleviate this curse of dimensionality has been to use additive\nmodels of \\emph{first order}, which model the regression function as a sum of\nindependent functions on each dimension. Though useful in controlling the\nvariance of the estimate, such models are often too restrictive in practical\nsettings. Between non-additive models which often have large variance and first\norder additive models which have large bias, there has been little work to\nexploit the trade-off in the middle via additive models of intermediate order.\nIn this work, we propose SALSA, which bridges this gap by allowing interactions\nbetween variables, but controls model capacity by limiting the order of\ninteractions. SALSA minimises the residual sum of squares with squared RKHS\nnorm penalties. Algorithmically, it can be viewed as Kernel Ridge Regression\nwith an additive kernel. When the regression function is additive, the excess\nrisk is only polynomial in dimension. Using the Girard-Newton formulae, we\nefficiently sum over a combinatorial number of terms in the additive expansion.\nVia a comparison on $15$ real datasets, we show that our method is competitive\nagainst $21$ other alternatives.","url_abs":"http://arxiv.org/abs/1602.00287v3","url_pdf":"http://arxiv.org/pdf/1602.00287v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"additive-approximations-in-high-dimensional","repo_url":"https://github.com/kirthevasank/local-poly-reg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null},{"paper_slug":"additive-approximations-in-high-dimensional","repo_url":"https://github.com/kirthevasank/salsa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"additive-models","task_name":"Additive models"},{"task_slug":"high","task_name":"Vocal Bursts Intensity Prediction"},{"task_slug":"regression-1","task_name":"regression"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1602.00287","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}