{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-hierarchical-semantic-segmentation","title":"Deep Hierarchical Semantic Segmentation","arxiv_id":"2203.14335","date":"2022-03-27","proceeding":"CVPR 2022 1","authors":["Liulei Li","Tianfei Zhou","Wenguan Wang","Jianwu Li","Yi Yang"],"abstract":"Humans are able to recognize structured relations in observation, allowing us to decompose complex scenes into simpler parts and abstract the visual world in multiple levels. However, such hierarchical reasoning ability of human perception remains largely unexplored in current literature of semantic segmentation. Existing work is often aware of flatten labels and predicts target classes exclusively for each pixel. In this paper, we instead address hierarchical semantic segmentation (HSS), which aims at structured, pixel-wise description of visual observation in terms of a class hierarchy. We devise HSSN, a general HSS framework that tackles two critical issues in this task: i) how to efficiently adapt existing hierarchy-agnostic segmentation networks to the HSS setting, and ii) how to leverage the hierarchy information to regularize HSS network learning. To address i), HSSN directly casts HSS as a pixel-wise multi-label classification task, only bringing minimal architecture change to current segmentation models. To solve ii), HSSN first explores inherent properties of the hierarchy as a training objective, which enforces segmentation predictions to obey the hierarchy structure. Further, with hierarchy-induced margin constraints, HSSN reshapes the pixel embedding space, so as to generate well-structured pixel representations and improve segmentation eventually. We conduct experiments on four semantic segmentation datasets (i.e., Mapillary Vistas 2.0, Cityscapes, LIP, and PASCAL-Person-Part), with different class hierarchies, segmentation network architectures and backbones, showing the generalization and superiority of HSSN.","url_abs":"https://arxiv.org/abs/2203.14335v2","url_pdf":"https://arxiv.org/pdf/2203.14335v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-hierarchical-semantic-segmentation","repo_url":"https://github.com/0liliulei/hieraseg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"deep-hierarchical-semantic-segmentation","repo_url":"https://github.com/lingorx/hieraseg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"deep-hierarchical-semantic-segmentation","repo_url":"https://github.com/tfzhou/protoseg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"multi-label-classification-2","task_name":"MUlTI-LABEL-ClASSIFICATION"},{"task_slug":"multi-label-classification","task_name":"Multi-Label Classification"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"aware","method_name":"AWARE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.14335","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.14335"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/0liliulei/hieraseg","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lingorx/hieraseg","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tfzhou/protoseg","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"listed":{"samples":5,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"59071c5dc8ab7c95","entry":"label_to_onehot","repo":"tfzhou/protoseg","repo_kind":"listed","path":"lib/models/modules/spatial_ocr_block.py","file_url":"https://github.com/tfzhou/protoseg/blob/HEAD/lib/models/modules/spatial_ocr_block.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59071c5dc8ab7c95"}},{"code_sha256_prefix":"f72d93bc5e399508","entry":"ModuleHelper","repo":"tfzhou/protoseg","repo_kind":"listed","path":"lib/models/modules/spatial_ocr_block.py","file_url":"https://github.com/tfzhou/protoseg/blob/HEAD/lib/models/modules/spatial_ocr_block.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f72d93bc5e399508"}},{"code_sha256_prefix":"cb39df60f3bc47c9","entry":"ObjectAttentionBlock2D","repo":"tfzhou/protoseg","repo_kind":"listed","path":"lib/models/modules/spatial_ocr_block.py","file_url":"https://github.com/tfzhou/protoseg/blob/HEAD/lib/models/modules/spatial_ocr_block.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb39df60f3bc47c9"}},{"code_sha256_prefix":"55d90fe00f511084","entry":"SpatialOCR_Module","repo":"tfzhou/protoseg","repo_kind":"listed","path":"lib/models/modules/spatial_ocr_block.py","file_url":"https://github.com/tfzhou/protoseg/blob/HEAD/lib/models/modules/spatial_ocr_block.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"55d90fe00f511084"}},{"code_sha256_prefix":"baf3518e7765c9a2","entry":"_ObjectAttentionBlock","repo":"tfzhou/protoseg","repo_kind":"listed","path":"lib/models/modules/spatial_ocr_block.py","file_url":"https://github.com/tfzhou/protoseg/blob/HEAD/lib/models/modules/spatial_ocr_block.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"baf3518e7765c9a2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}