{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/task-aligned-part-aware-panoptic-segmentation-1","title":"Task-aligned Part-aware Panoptic Segmentation through Joint Object-Part Representations","arxiv_id":"2406.10114","date":"2024-06-14","proceeding":"CVPR 2024 1","authors":["Daan de Geus","Gijs Dubbelman"],"abstract":"Part-aware panoptic segmentation (PPS) requires (a) that each foreground object and background region in an image is segmented and classified, and (b) that all parts within foreground objects are segmented, classified and linked to their parent object. Existing methods approach PPS by separately conducting object-level and part-level segmentation. However, their part-level predictions are not linked to individual parent objects. Therefore, their learning objective is not aligned with the PPS task objective, which harms the PPS performance. To solve this, and make more accurate PPS predictions, we propose Task-Aligned Part-aware Panoptic Segmentation (TAPPS). This method uses a set of shared queries to jointly predict (a) object-level segments, and (b) the part-level segments within those same objects. As a result, TAPPS learns to predict part-level segments that are linked to individual parent objects, aligning the learning objective with the task objective, and allowing TAPPS to leverage joint object-part representations. With experiments, we show that TAPPS considerably outperforms methods that predict objects and parts separately, and achieves new state-of-the-art PPS results.","url_abs":"https://arxiv.org/abs/2406.10114v1","url_pdf":"https://arxiv.org/pdf/2406.10114v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"task-aligned-part-aware-panoptic-segmentation-1","repo_url":"https://github.com/tue-mps/tapps","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"panoptic-segmentation","task_name":"Panoptic Segmentation"},{"task_slug":"part-level-panoptic-segmentation","task_name":"Part-aware Panoptic Segmentation"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/part-aware-panoptic-segmentation-on","task":"Part-aware Panoptic Segmentation","dataset":"Cityscapes Panoptic Parts","model":"TAPPS (Swin-B, COCO pre-training)","rank_in_archive_order":1,"of":4,"metrics":{"PartPQ":"64.8"},"uses_additional_data":false},{"leaderboard":"/sota/part-aware-panoptic-segmentation-on-pascal","task":"Part-aware Panoptic Segmentation","dataset":"Pascal Panoptic Parts","model":"TAPPS (Swin-B, COCO pre-training)","rank_in_archive_order":1,"of":4,"metrics":{"PartPQ":"60.4"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.10114","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.10114"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tue-mps/tapps","reach":null}],"summary":{"ran":1,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"88342bac1e676835","entry":"MLP","repo":"tue-mps/tapps","repo_kind":"official","path":"tapps/modeling/transformer_decoder/part_decoder.py","file_url":"https://github.com/tue-mps/tapps/blob/HEAD/tapps/modeling/transformer_decoder/part_decoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"88342bac1e676835"}},{"code_sha256_prefix":"d719e97ac4f28c87","entry":"PartDecoder","repo":"tue-mps/tapps","repo_kind":"official","path":"tapps/modeling/transformer_decoder/part_decoder.py","file_url":"https://github.com/tue-mps/tapps/blob/HEAD/tapps/modeling/transformer_decoder/part_decoder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d719e97ac4f28c87"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}