Papers › Maintaining Discrimination and Fairness in Class Incremental Learning
Maintaining Discrimination and Fairness in Class Incremental Learning
Bowen Zhao, Xi Xiao, Guojun Gan, Bin Zhang, Shu-Tao Xia
Deep neural networks (DNNs) have been applied in class incremental learning, which aims to solve common real-world problems of learning new classes continually. One drawback of standard DNNs is that they are prone to catastrophic forgetting. Knowledge distillation (KD) is a commonly used technique to alleviate this problem. In this paper, we demonstrate it can indeed help the model to output more discriminative results within old classes. However, it cannot alleviate the problem that the model tends to classify objects into new classes, causing the positive effect of KD to be hidden and limited. We observed that an important factor causing catastrophic forgetting is that the weights in the last fully connected (FC) layer are highly biased in class incremental learning. In this paper, we propose a simple and effective solution motivated by the aforementioned observations to address catastrophic forgetting. Firstly, we utilize KD to maintain the discrimination within old classes. Then, to further maintain the fairness between old classes and new classes, we propose Weight Aligning (WA) that corrects the biased weights in the FC layer after normal training process. Unlike previous work, WA does not require any extra parameters or a validation set in advance, as it utilizes the information provided by the biased weights themselves. The proposed method is evaluated on ImageNet-1000, ImageNet-100, and CIFAR-100 under various settings. Experimental results show that the proposed method can effectively alleviate catastrophic forgetting and significantly outperform state-of-the-art methods.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Incremental Learning | CIFAR-100-B0(5steps of 20 classes) | WA | Average Incremental Accuracy | 72.81 | #6 of 10 | Archive leaderboard | report |
| Incremental Learning | ImageNet - 10 steps | WA | # M Params | 11.68 | #7 of 10 | Archive leaderboard | report |
| Incremental Learning | ImageNet - 10 steps | WA | Average Incremental Accuracy | 65.67 | #7 of 10 | Archive leaderboard | report |
| Incremental Learning | ImageNet - 10 steps | WA | Average Incremental Accuracy Top-5 | 86.60 | #7 of 10 | Archive leaderboard | report |
| Incremental Learning | ImageNet - 10 steps | WA | Final Accuracy | 55.60 | #7 of 10 | Archive leaderboard | report |
| Incremental Learning | ImageNet - 10 steps | WA | Final Accuracy Top-5 | 81.10 | #7 of 10 | Archive leaderboard | report |
| Incremental Learning | ImageNet100 - 10 steps | WA | # M Params | 11.22 | #9 of 13 | Archive leaderboard | report |
| Incremental Learning | ImageNet100 - 10 steps | WA | Average Incremental Accuracy Top-5 | 91.00 | #9 of 13 | Archive leaderboard | report |
| Incremental Learning | ImageNet100 - 10 steps | WA | Final Accuracy Top-5 | 84.10 | #9 of 13 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections