Papers › Supervised Models Can Generalize Also When Trained on Random Label
Supervised Models Can Generalize Also When Trained on Random Label
Oskar Allerbo, Thomas B. Schön
The success of unsupervised learning raises the question of whether also supervised models can be trained without using the information in the output y. In this paper, we demonstrate that this is indeed possible. The key step is to formulate the model as a smoother, i.e. on the form f̂=Sy, and to construct the smoother matrix S independently of y, e.g. by training on random labels. We present a simple model selection criterion based on the distribution of the out-of-sample predictions and show that, in contrast to cross-validation, this criterion can be used also without access to y. We demonstrate on real and synthetic data that y-free trained versions of linear and kernel ridge regression, smoothing splines, and neural networks perform similarly to their standard, y-based, versions and, most importantly, significantly better than random guessing.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
No leaderboard rows for this paper in the archive.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections