{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/faster-kernel-ridge-regression-using","title":"Faster Kernel Ridge Regression Using Sketching and Preconditioning","arxiv_id":"1611.03220","date":"2016-11-10","proceeding":null,"authors":["Haim Avron","Kenneth L. Clarkson","David P. Woodruff"],"abstract":"Kernel Ridge Regression (KRR) is a simple yet powerful technique for\nnon-parametric regression whose computation amounts to solving a linear system.\nThis system is usually dense and highly ill-conditioned. In addition, the\ndimensions of the matrix are the same as the number of data points, so direct\nmethods are unrealistic for large-scale datasets. In this paper, we propose a\npreconditioning technique for accelerating the solution of the aforementioned\nlinear system. The preconditioner is based on random feature maps, such as\nrandom Fourier features, which have recently emerged as a powerful technique\nfor speeding up and scaling the training of kernel-based methods, such as\nkernel ridge regression, by resorting to approximations. However, random\nfeature maps only provide crude approximations to the kernel function, so\ndelivering state-of-the-art results by directly solving the approximated system\nrequires the number of random features to be very large. We show that random\nfeature maps can be much more effective in forming preconditioners, since under\ncertain conditions a not-too-large number of random features is sufficient to\nyield an effective preconditioner. We empirically evaluate our method and show\nit is highly effective for datasets of up to one million training examples.","url_abs":"http://arxiv.org/abs/1611.03220v4","url_pdf":"http://arxiv.org/pdf/1611.03220v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"faster-kernel-ridge-regression-using","repo_url":"https://github.com/tengandreaxu/fabr","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"regression-1","task_name":"regression"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1611.03220","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}