{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/segsort-segmentation-by-discriminative","title":"SegSort: Segmentation by Discriminative Sorting of Segments","arxiv_id":"1910.06962","date":"2019-10-15","proceeding":"ICCV 2019 10","authors":["Jyh-Jing Hwang","Stella X. Yu","Jianbo Shi","Maxwell D. Collins","Tien-Ju Yang","Xiao Zhang","Liang-Chieh Chen"],"abstract":"Almost all existing deep learning approaches for semantic segmentation tackle this task as a pixel-wise classification problem. Yet humans understand a scene not in terms of pixels, but by decomposing it into perceptual groups and structures that are the basic building blocks of recognition. This motivates us to propose an end-to-end pixel-wise metric learning approach that mimics this process. In our approach, the optimal visual representation determines the right segmentation within individual images and associates segments with the same semantic classes across images. The core visual learning problem is therefore to maximize the similarity within segments and minimize the similarity between segments. Given a model trained this way, inference is performed consistently by extracting pixel-wise embeddings and clustering, with the semantic label determined by the majority vote of its nearest neighbors from an annotated set. As a result, we present the SegSort, as a first attempt using deep learning for unsupervised semantic segmentation, achieving $76\\%$ performance of its supervised counterpart. When supervision is available, SegSort shows consistent improvements over conventional approaches based on pixel-wise softmax training. Additionally, our approach produces more precise boundaries and consistent region predictions. The proposed SegSort further produces an interpretable result, as each choice of label can be easily understood from the retrieved nearest segments.","url_abs":"https://arxiv.org/abs/1910.06962v2","url_pdf":"https://arxiv.org/pdf/1910.06962v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"segsort-segmentation-by-discriminative","repo_url":"https://github.com/jyhjinghwang/segsort","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"clustering","task_name":"Clustering"},{"task_slug":"metric-learning","task_name":"Metric Learning"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"unsupervised-semantic-segmentation","task_name":"Unsupervised Semantic Segmentation"}],"methods":[{"method_slug":"segsort","method_name":"SegSort"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[{"slug":"segsort","name":"SegSort","full_name":"Segment Sorting"}],"results":[{"leaderboard":"/sota/unsupervised-semantic-segmentation-on-pascal-1","task":"Unsupervised Semantic Segmentation","dataset":"PASCAL VOC 2012 val","model":"SegSort (Edges)","rank_in_archive_order":12,"of":12,"metrics":{"Clustering [mIoU]":"-","Linear Classifier [mIoU]":"55.86 (KNN)"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1910.06962","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1910.06962"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jyhjinghwang/segsort","reach":null}],"summary":{"ran_honours":1},"by_repo_kind":{"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9c1264338253562e","entry":"parse_commastr","repo":"jyhjinghwang/segsort","repo_kind":"listed","path":"pyscripts/inference/inference_segsort.py","file_url":"https://github.com/jyhjinghwang/segsort/blob/HEAD/pyscripts/inference/inference_segsort.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9c1264338253562e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}