{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-metric-learning-for-open-world-semantic","title":"Deep Metric Learning for Open World Semantic Segmentation","arxiv_id":"2108.04562","date":"2021-08-10","proceeding":"ICCV 2021 10","authors":["Jun Cen","Peng Yun","Junhao Cai","Michael Yu Wang","Ming Liu"],"abstract":"Classical close-set semantic segmentation networks have limited ability to detect out-of-distribution (OOD) objects, which is important for safety-critical applications such as autonomous driving. Incrementally learning these OOD objects with few annotations is an ideal way to enlarge the knowledge base of the deep learning models. In this paper, we propose an open world semantic segmentation system that includes two modules: (1) an open-set semantic segmentation module to detect both in-distribution and OOD objects. (2) an incremental few-shot learning module to gradually incorporate those OOD objects into its existing knowledge base. This open world semantic segmentation system behaves like a human being, which is able to identify OOD objects and gradually learn them with corresponding supervision. We adopt the Deep Metric Learning Network (DMLNet) with contrastive clustering to implement open-set semantic segmentation. Compared to other open-set semantic segmentation methods, our DMLNet achieves state-of-the-art performance on three challenging open-set semantic segmentation datasets without using additional data or generative models. On this basis, two incremental few-shot learning methods are further proposed to progressively improve the DMLNet with the annotations of OOD objects.","url_abs":"https://arxiv.org/abs/2108.04562v1","url_pdf":"https://arxiv.org/pdf/2108.04562v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-metric-learning-for-open-world-semantic","repo_url":"https://github.com/Jun-CEN/Open-World-Semantic-Segmentation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"},{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"metric-learning","task_name":"Metric Learning"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2108.04562","atlas_url":"https://app.syntology.ai/?focus=2108.04562","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2108.04562"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Jun-CEN/Open-World-Semantic-Segmentation","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"ran_honours":1,"unverified":5},"by_repo_kind":{"listed":{"samples":10,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"345e436021e6eb22","entry":"Normalization","repo":"Jun-CEN/Open-World-Semantic-Segmentation","repo_kind":"listed","path":"DeepLabV3Plus-Pytorch/main_distillation.py","file_url":"https://github.com/Jun-CEN/Open-World-Semantic-Segmentation/blob/HEAD/DeepLabV3Plus-Pytorch/main_distillation.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"345e436021e6eb22"}},{"code_sha256_prefix":"f661b8566492d18b","entry":"estimator","repo":"Jun-CEN/Open-World-Semantic-Segmentation","repo_kind":"listed","path":"DeepLabV3Plus-Pytorch/Read_img.py","file_url":"https://github.com/Jun-CEN/Open-World-Semantic-Segmentation/blob/HEAD/DeepLabV3Plus-Pytorch/Read_img.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f661b8566492d18b"}},{"code_sha256_prefix":"7242da839b9822be","entry":"fpr_and_fdr_at_recall","repo":"Jun-CEN/Open-World-Semantic-Segmentation","repo_kind":"listed","path":"anomaly/anom_utils.py","file_url":"https://github.com/Jun-CEN/Open-World-Semantic-Segmentation/blob/HEAD/anomaly/anom_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7242da839b9822be"}},{"code_sha256_prefix":"b9b6c36f7a3e7e67","entry":"get_measures","repo":"Jun-CEN/Open-World-Semantic-Segmentation","repo_kind":"listed","path":"anomaly/anom_utils.py","file_url":"https://github.com/Jun-CEN/Open-World-Semantic-Segmentation/blob/HEAD/anomaly/anom_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b9b6c36f7a3e7e67"}},{"code_sha256_prefix":"d4acb3120a027622","entry":"stable_cumsum","repo":"Jun-CEN/Open-World-Semantic-Segmentation","repo_kind":"listed","path":"anomaly/anom_utils.py","file_url":"https://github.com/Jun-CEN/Open-World-Semantic-Segmentation/blob/HEAD/anomaly/anom_utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d4acb3120a027622"}},{"code_sha256_prefix":"de2f19be459091e2","entry":"Certainty","repo":"Jun-CEN/Open-World-Semantic-Segmentation","repo_kind":"listed","path":"DeepLabV3Plus-Pytorch/main_distillation.py","file_url":"https://github.com/Jun-CEN/Open-World-Semantic-Segmentation/blob/HEAD/DeepLabV3Plus-Pytorch/main_distillation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de2f19be459091e2"}},{"code_sha256_prefix":"ceca25f872bfae7f","entry":"convert_to_separable_conv","repo":"Jun-CEN/Open-World-Semantic-Segmentation","repo_kind":"listed","path":"DeepLabV3Plus-Pytorch/network/utils.py","file_url":"https://github.com/Jun-CEN/Open-World-Semantic-Segmentation/blob/HEAD/DeepLabV3Plus-Pytorch/network/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ceca25f872bfae7f"}},{"code_sha256_prefix":"395489c4f58f7d7b","entry":"deeplabv3_mobilenet","repo":"Jun-CEN/Open-World-Semantic-Segmentation","repo_kind":"listed","path":"DeepLabV3Plus-Pytorch/network/modeling.py","file_url":"https://github.com/Jun-CEN/Open-World-Semantic-Segmentation/blob/HEAD/DeepLabV3Plus-Pytorch/network/modeling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"395489c4f58f7d7b"}},{"code_sha256_prefix":"ced2abecfa48dbf8","entry":"deeplabv3_resnet101","repo":"Jun-CEN/Open-World-Semantic-Segmentation","repo_kind":"listed","path":"DeepLabV3Plus-Pytorch/network/modeling.py","file_url":"https://github.com/Jun-CEN/Open-World-Semantic-Segmentation/blob/HEAD/DeepLabV3Plus-Pytorch/network/modeling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ced2abecfa48dbf8"}},{"code_sha256_prefix":"4de0af0762139e5b","entry":"deeplabv3_resnet50","repo":"Jun-CEN/Open-World-Semantic-Segmentation","repo_kind":"listed","path":"DeepLabV3Plus-Pytorch/network/modeling.py","file_url":"https://github.com/Jun-CEN/Open-World-Semantic-Segmentation/blob/HEAD/DeepLabV3Plus-Pytorch/network/modeling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4de0af0762139e5b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}