{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/large-scale-incremental-learning-1","title":"Large Scale Incremental Learning","arxiv_id":"1905.13260","date":"2019-05-30","proceeding":"CVPR 2019 6","authors":["Yue Wu","Yinpeng Chen","Lijuan Wang","Yuancheng Ye","Zicheng Liu","Yandong Guo","Yun Fu"],"abstract":"Modern machine learning suffers from catastrophic forgetting when learning new classes incrementally. The performance dramatically degrades due to the missing data of old classes. Incremental learning methods have been proposed to retain the knowledge acquired from the old classes, by using knowledge distilling and keeping a few exemplars from the old classes. However, these methods struggle to scale up to a large number of classes. We believe this is because of the combination of two factors: (a) the data imbalance between the old and new classes, and (b) the increasing number of visually similar classes. Distinguishing between an increasing number of visually similar classes is particularly challenging, when the training data is unbalanced. We propose a simple and effective method to address this data imbalance issue. We found that the last fully connected layer has a strong bias towards the new classes, and this bias can be corrected by a linear model. With two bias parameters, our method performs remarkably well on two large datasets: ImageNet (1000 classes) and MS-Celeb-1M (10000 classes), outperforming the state-of-the-art algorithms by 11.1% and 13.2% respectively.","url_abs":"https://arxiv.org/abs/1905.13260v1","url_pdf":"https://arxiv.org/pdf/1905.13260v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"large-scale-incremental-learning-1","repo_url":"https://github.com/g-u-n/pycil","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"large-scale-incremental-learning-1","repo_url":"https://github.com/sairin1202/BIC","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"large-scale-incremental-learning-1","repo_url":"https://github.com/ContinualAI/avalanche","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"large-scale-incremental-learning-1","repo_url":"https://github.com/aimagelab/mammoth","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"large-scale-incremental-learning-1","repo_url":"https://github.com/mmasana/FACIL","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"class-incremental-learning","task_name":"Class Incremental Learning"},{"task_slug":"incremental-learning","task_name":"Incremental Learning"},{"task_slug":"class-incremental-learning-1","task_name":"class-incremental learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/incremental-learning-on-cifar-100-50-classes-2","task":"Incremental Learning","dataset":"CIFAR-100 - 50 classes + 10 steps of 5 classes","model":"BiC","rank_in_archive_order":12,"of":13,"metrics":{"Average Incremental Accuracy":"53.21"},"uses_additional_data":false},{"leaderboard":"/sota/incremental-learning-on-cifar-100-50-classes-1","task":"Incremental Learning","dataset":"CIFAR-100 - 50 classes + 25 steps of 2 classes","model":"BiC","rank_in_archive_order":5,"of":5,"metrics":{"Average Incremental Accuracy":"48.96"},"uses_additional_data":false},{"leaderboard":"/sota/incremental-learning-on-cifar-100-50-classes-3","task":"Incremental Learning","dataset":"CIFAR-100 - 50 classes + 5 steps of 10 classes","model":"BiC","rank_in_archive_order":13,"of":15,"metrics":{"Average Incremental Accuracy":"56.86"},"uses_additional_data":false},{"leaderboard":"/sota/incremental-learning-on-cifar-100-50-classes","task":"Incremental Learning","dataset":"CIFAR-100 - 50 classes + 50 steps of 1 class","model":"BiC","rank_in_archive_order":2,"of":2,"metrics":{"Average Incremental Accuracy":"47.09"},"uses_additional_data":false},{"leaderboard":"/sota/incremental-learning-on-cifar-100-b0-5steps","task":"Incremental Learning","dataset":"CIFAR-100-B0(5steps of 20 classes)","model":"BiC","rank_in_archive_order":5,"of":10,"metrics":{"Average Incremental Accuracy":"73.10"},"uses_additional_data":false},{"leaderboard":"/sota/incremental-learning-on-imagenet-10-steps","task":"Incremental Learning","dataset":"ImageNet - 10 steps","model":"BiC","rank_in_archive_order":9,"of":10,"metrics":{"# M Params":"11.68","Average Incremental Accuracy Top-5":"84.00","Final Accuracy Top-5":"73.20"},"uses_additional_data":false},{"leaderboard":"/sota/incremental-learning-on-imagenet-100-50","task":"Incremental Learning","dataset":"ImageNet-100 - 50 classes + 50 steps of 1 class","model":"BiC","rank_in_archive_order":2,"of":2,"metrics":{"Average Incremental Accuracy":"46.49"},"uses_additional_data":false},{"leaderboard":"/sota/incremental-learning-on-imagenet100-10-steps","task":"Incremental Learning","dataset":"ImageNet100 - 10 steps","model":"BiC","rank_in_archive_order":10,"of":13,"metrics":{"# M Params":"11.22","Average Incremental Accuracy Top-5":"90.60","Final Accuracy Top-5":"84.40"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1905.13260","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1905.13260"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aimagelab/mammoth","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sairin1202/BIC","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ContinualAI/avalanche","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mmasana/FACIL","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/g-u-n/pycil","reach":null}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}