Papers › Fast Robust Kernel Regression through Sign Gradient Descent with Early Stopping
Fast Robust Kernel Regression through Sign Gradient Descent with Early Stopping
Oskar Allerbo
Kernel ridge regression, KRR, is a generalization of linear ridge regression that is non-linear in the data, but linear in the model parameters. Here, we introduce an equivalent formulation of the objective function of KRR, which opens up for replacing the ridge penalty with the ℓ_∞ and ℓ₁ penalties. Using the ℓ_∞ and ℓ₁ penalties, we obtain robust and sparse kernel regression, respectively. We study the similarities between explicitly regularized kernel regression and the solutions obtained by early stopping of iterative gradient-based methods, where we connect ℓ_∞ regularization to sign gradient descent, ℓ₁ regularization to forward stagewise regression (also known as coordinate descent), and ℓ₂ regularization to gradient descent, and, in the last case, theoretically bound for the differences. We exploit the close relations between ℓ_∞ regularization and sign gradient descent, and between ℓ₁ regularization and coordinate descent to propose computationally efficient methods for robust and sparse kernel regression. We finally compare robust kernel regression through sign gradient descent to existing methods for robust kernel regression on five real data sets, demonstrating that our method is one to two orders of magnitude faster, without compromised accuracy.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
No leaderboard rows for this paper in the archive.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections