{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ssul-semantic-segmentation-with-unknown-label","title":"SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning","arxiv_id":"2106.11562","date":"2021-06-22","proceeding":"NeurIPS 2021 12","authors":["Sungmin Cha","Beomyoung Kim","Youngjoon Yoo","Taesup Moon"],"abstract":"This paper introduces a solid state-of-the-art baseline for a class-incremental semantic segmentation (CISS) problem. While the recent CISS algorithms utilize variants of the knowledge distillation (KD) technique to tackle the problem, they failed to fully address the critical challenges in CISS causing the catastrophic forgetting; the semantic drift of the background class and the multi-label prediction issue. To better address these challenges, we propose a new method, dubbed SSUL-M (Semantic Segmentation with Unknown Label with Memory), by carefully combining techniques tailored for semantic segmentation. Specifically, we claim three main contributions. (1) defining unknown classes within the background class to help to learn future classes (help plasticity), (2) freezing backbone network and past classifiers with binary cross-entropy loss and pseudo-labeling to overcome catastrophic forgetting (help stability), and (3) utilizing tiny exemplar memory for the first time in CISS to improve both plasticity and stability. The extensively conducted experiments show the effectiveness of our method, achieving significantly better performance than the recent state-of-the-art baselines on the standard benchmark datasets. Furthermore, we justify our contributions with thorough ablation analyses and discuss different natures of the CISS problem compared to the traditional class-incremental learning targeting classification. The official code is available at https://github.com/clovaai/SSUL.","url_abs":"https://arxiv.org/abs/2106.11562v3","url_pdf":"https://arxiv.org/pdf/2106.11562v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ssul-semantic-segmentation-with-unknown-label","repo_url":"https://github.com/clovaai/SSUL","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"class-incremental-learning","task_name":"Class Incremental Learning"},{"task_slug":"class-incremental-semantic-segmentation","task_name":"Class-Incremental Semantic Segmentation"},{"task_slug":"continual-semantic-segmentation","task_name":"Continual Semantic Segmentation"},{"task_slug":"disjoint-10-1","task_name":"Disjoint 10-1"},{"task_slug":"disjoint-15-1","task_name":"Disjoint 15-1"},{"task_slug":"disjoint-15-5","task_name":"Disjoint 15-5"},{"task_slug":"incremental-learning","task_name":"Incremental Learning"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"overlapped-10-1","task_name":"Overlapped 10-1"},{"task_slug":"overlapped-100-5","task_name":"Overlapped 100-5"},{"task_slug":"overlapped-100-50","task_name":"Overlapped 100-50"},{"task_slug":"overlapped-15-1","task_name":"Overlapped 15-1"},{"task_slug":"overlapped-15-5","task_name":"Overlapped 15-5"},{"task_slug":null,"task_name":"Overlapped 19-1"},{"task_slug":"overlapped-50-50","task_name":"Overlapped 50-50"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"class-incremental-learning-1","task_name":"class-incremental learning"}],"methods":[{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/disjoint-10-1-on-pascal-voc-2012","task":"Disjoint 10-1","dataset":"PASCAL VOC 2012","model":"SSUL-M","rank_in_archive_order":1,"of":8,"metrics":{"mIoU":"53.50"},"uses_additional_data":false},{"leaderboard":"/sota/disjoint-10-1-on-pascal-voc-2012","task":"Disjoint 10-1","dataset":"PASCAL VOC 2012","model":"SSUL","rank_in_archive_order":2,"of":8,"metrics":{"mIoU":"50.87"},"uses_additional_data":false},{"leaderboard":"/sota/disjoint-15-1-on-pascal-voc-2012","task":"Disjoint 15-1","dataset":"PASCAL VOC 2012","model":"SSUL-M","rank_in_archive_order":2,"of":9,"metrics":{"mIoU":"68.58"},"uses_additional_data":false},{"leaderboard":"/sota/disjoint-15-1-on-pascal-voc-2012","task":"Disjoint 15-1","dataset":"PASCAL VOC 2012","model":"SSUL","rank_in_archive_order":3,"of":9,"metrics":{"mIoU":"64.01"},"uses_additional_data":false},{"leaderboard":"/sota/disjoint-15-5-on-pascal-voc-2012","task":"Disjoint 15-5","dataset":"PASCAL VOC 2012","model":"SSUL-M","rank_in_archive_order":2,"of":9,"metrics":{"Mean IoU":"69.83"},"uses_additional_data":false},{"leaderboard":"/sota/disjoint-15-5-on-pascal-voc-2012","task":"Disjoint 15-5","dataset":"PASCAL VOC 2012","model":"SSUL","rank_in_archive_order":3,"of":9,"metrics":{"Mean IoU":"69.10"},"uses_additional_data":false},{"leaderboard":"/sota/overlapped-10-1-on-pascal-voc-2012","task":"Overlapped 10-1","dataset":"PASCAL VOC 2012","model":"SSUL-M","rank_in_archive_order":3,"of":13,"metrics":{"mIoU":"64.12"},"uses_additional_data":false},{"leaderboard":"/sota/overlapped-10-1-on-pascal-voc-2012","task":"Overlapped 10-1","dataset":"PASCAL VOC 2012","model":"SSUL","rank_in_archive_order":6,"of":13,"metrics":{"mIoU":"59.25"},"uses_additional_data":false},{"leaderboard":"/sota/overlapped-100-5-on-ade20k","task":"Overlapped 100-5","dataset":"ADE20K","model":"SSUL-M","rank_in_archive_order":2,"of":8,"metrics":{"mIoU":"34.56"},"uses_additional_data":false},{"leaderboard":"/sota/overlapped-100-5-on-ade20k","task":"Overlapped 100-5","dataset":"ADE20K","model":"SSUL","rank_in_archive_order":3,"of":8,"metrics":{"mIoU":"32.48"},"uses_additional_data":false},{"leaderboard":"/sota/overlapped-100-50-on-ade20k","task":"Overlapped 100-50","dataset":"ADE20K","model":"SSUL-M","rank_in_archive_order":4,"of":7,"metrics":{"mIoU":"34.37"},"uses_additional_data":false},{"leaderboard":"/sota/overlapped-100-50-on-ade20k","task":"Overlapped 100-50","dataset":"ADE20K","model":"SSUL","rank_in_archive_order":5,"of":7,"metrics":{"mIoU":"33.58"},"uses_additional_data":false},{"leaderboard":"/sota/overlapped-15-1-on-pascal-voc-2012","task":"Overlapped 15-1","dataset":"PASCAL VOC 2012","model":"SSUL-M","rank_in_archive_order":4,"of":13,"metrics":{"mIoU":"71.37"},"uses_additional_data":false},{"leaderboard":"/sota/overlapped-15-1-on-pascal-voc-2012","task":"Overlapped 15-1","dataset":"PASCAL VOC 2012","model":"SSUL","rank_in_archive_order":5,"of":13,"metrics":{"mIoU":"67.61"},"uses_additional_data":false},{"leaderboard":"/sota/overlapped-15-5-on-pascal-voc-2012","task":"Overlapped 15-5","dataset":"PASCAL VOC 2012","model":"SSUL-M","rank_in_archive_order":4,"of":13,"metrics":{"Mean IoU (val)":"73.02"},"uses_additional_data":false},{"leaderboard":"/sota/overlapped-15-5-on-pascal-voc-2012","task":"Overlapped 15-5","dataset":"PASCAL VOC 2012","model":"SSUL","rank_in_archive_order":7,"of":13,"metrics":{"Mean IoU (val)":"71.22"},"uses_additional_data":false},{"leaderboard":"/sota/overlapped-50-50-on-ade20k","task":"Overlapped 50-50","dataset":"ADE20K","model":"SSUL-M","rank_in_archive_order":5,"of":7,"metrics":{"mIoU":"29.77"},"uses_additional_data":false},{"leaderboard":"/sota/overlapped-50-50-on-ade20k","task":"Overlapped 50-50","dataset":"ADE20K","model":"SSUL","rank_in_archive_order":6,"of":7,"metrics":{"mIoU":"29.56"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.11562","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.11562"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/clovaai/SSUL","reach":null}],"summary":{"ran":6,"unverified":6},"by_repo_kind":{"official":{"samples":12,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"ca5a01506bf190fc","entry":"ASPP","repo":"clovaai/SSUL","repo_kind":"official","path":"network/modeling.py","file_url":"https://github.com/clovaai/SSUL/blob/HEAD/network/modeling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"ca5a01506bf190fc"}},{"code_sha256_prefix":"3704384e62d46e83","entry":"ASPPConv","repo":"clovaai/SSUL","repo_kind":"official","path":"network/modeling.py","file_url":"https://github.com/clovaai/SSUL/blob/HEAD/network/modeling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3704384e62d46e83"}},{"code_sha256_prefix":"58aa3ee6bb3dd9a4","entry":"ASPPPooling","repo":"clovaai/SSUL","repo_kind":"official","path":"network/modeling.py","file_url":"https://github.com/clovaai/SSUL/blob/HEAD/network/modeling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"58aa3ee6bb3dd9a4"}},{"code_sha256_prefix":"9917e2638ee6fb29","entry":"DeepLabHead","repo":"clovaai/SSUL","repo_kind":"official","path":"network/modeling.py","file_url":"https://github.com/clovaai/SSUL/blob/HEAD/network/modeling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"9917e2638ee6fb29"}},{"code_sha256_prefix":"8d2daac3bc9124e6","entry":"IntermediateLayerGetter","repo":"clovaai/SSUL","repo_kind":"official","path":"network/modeling.py","file_url":"https://github.com/clovaai/SSUL/blob/HEAD/network/modeling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"8d2daac3bc9124e6"}},{"code_sha256_prefix":"96c800659dca7d1e","entry":"_SimpleSegmentationModel","repo":"clovaai/SSUL","repo_kind":"official","path":"network/modeling.py","file_url":"https://github.com/clovaai/SSUL/blob/HEAD/network/modeling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"96c800659dca7d1e"}},{"code_sha256_prefix":"3cab456d98fae710","entry":"DeepLabHeadV3Plus","repo":"clovaai/SSUL","repo_kind":"official","path":"network/modeling.py","file_url":"https://github.com/clovaai/SSUL/blob/HEAD/network/modeling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3cab456d98fae710"}},{"code_sha256_prefix":"aefeef7f716ecb9d","entry":"DeepLabV3","repo":"clovaai/SSUL","repo_kind":"official","path":"network/modeling.py","file_url":"https://github.com/clovaai/SSUL/blob/HEAD/network/modeling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"aefeef7f716ecb9d"}},{"code_sha256_prefix":"c09c3bf8b03dab46","entry":"_load_model","repo":"clovaai/SSUL","repo_kind":"official","path":"network/modeling.py","file_url":"https://github.com/clovaai/SSUL/blob/HEAD/network/modeling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c09c3bf8b03dab46"}},{"code_sha256_prefix":"3abf7e51960c13a7","entry":"_segm_mobilenet","repo":"clovaai/SSUL","repo_kind":"official","path":"network/modeling.py","file_url":"https://github.com/clovaai/SSUL/blob/HEAD/network/modeling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3abf7e51960c13a7"}},{"code_sha256_prefix":"3187feb03a45e3a4","entry":"_segm_resnet","repo":"clovaai/SSUL","repo_kind":"official","path":"network/modeling.py","file_url":"https://github.com/clovaai/SSUL/blob/HEAD/network/modeling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3187feb03a45e3a4"}},{"code_sha256_prefix":"0c67be585d1b2be6","entry":"deeplabv3plus_resnet50","repo":"clovaai/SSUL","repo_kind":"official","path":"network/modeling.py","file_url":"https://github.com/clovaai/SSUL/blob/HEAD/network/modeling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"0c67be585d1b2be6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}