Papers › Learning to Reweight Examples for Robust Deep Learning

Learning to Reweight Examples for Robust Deep Learning

24 Mar 2018ICML 2018 7arXiv:1803.09050archive 2025-07-28

Mengye Ren, Wenyuan Zeng, Bin Yang, Raquel Urtasun

Deep neural networks have been shown to be very powerful modeling tools for many supervised learning tasks involving complex input patterns. However, they can also easily overfit to training set biases and label noises. In addition to various regularizers, example reweighting algorithms are popular solutions to these problems, but they require careful tuning of additional hyperparameters, such as example mining schedules and regularization hyperparameters. In contrast to past reweighting methods, which typically consist of functions of the cost value of each example, in this work we propose a novel meta-learning algorithm that learns to assign weights to training examples based on their gradient directions. To determine the example weights, our method performs a meta gradient descent step on the current mini-batch example weights (which are initialized from zero) to minimize the loss on a clean unbiased validation set. Our proposed method can be easily implemented on any type of deep network, does not require any additional hyperparameter tuning, and achieves impressive performance on class imbalance and corrupted label problems where only a small amount of clean validation data is available.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

uber-research/learning-to-reweight-examples officialmentioned in papermentioned on GitHubtfNOASSERTIONSyntology: not harvested report
JiyangZhang/learning-to-reweight-in-relation-extraion mentioned on GitHubpytorchSyntology: not harvested report
awasthiabhijeet/Learning-From-Rules mentioned on GitHubtfApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Learning from Rules Generalizing Labeled Exemplars (arXiv:2004.06025; that paper's own run record: 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 1 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 6 unverified); nothing checks that they implement this paper's method. report
danieltan07/learning-to-reweight-examples mentioned on GitHubpytorchSyntology: not harvested report
microsoft/MWSS mentioned on GitHubpytorchNOASSERTIONSyntology: not harvested report
pfnet-research/robust_estimation mentioned on GitHubBSD-3-ClauseSyntology: not harvested report
tanyuqian/learning-data-manipulation mentioned on GitHubpytorchSyntology: not harvested report
zhangyuwangumass/Trajectory-Reweighted-Sampler mentioned on GitHubpytorchSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Deep LearningMeta-Learning

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections