{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-unified-transformer-framework-for-group","title":"A Unified Transformer Framework for Group-based Segmentation: Co-Segmentation, Co-Saliency Detection and Video Salient Object Detection","arxiv_id":"2203.04708","date":"2022-03-09","proceeding":null,"authors":["Yukun Su","Jingliang Deng","Ruizhou Sun","Guosheng Lin","Qingyao Wu"],"abstract":"Humans tend to mine objects by learning from a group of images or several frames of video since we live in a dynamic world. In the computer vision area, many researches focus on co-segmentation (CoS), co-saliency detection (CoSD) and video salient object detection (VSOD) to discover the co-occurrent objects. However, previous approaches design different networks on these similar tasks separately, and they are difficult to apply to each other, which lowers the upper bound of the transferability of deep learning frameworks. Besides, they fail to take full advantage of the cues among inter- and intra-feature within a group of images. In this paper, we introduce a unified framework to tackle these issues, term as UFO (Unified Framework for Co-Object Segmentation). Specifically, we first introduce a transformer block, which views the image feature as a patch token and then captures their long-range dependencies through the self-attention mechanism. This can help the network to excavate the patch structured similarities among the relevant objects. Furthermore, we propose an intra-MLP learning module to produce self-mask to enhance the network to avoid partial activation. Extensive experiments on four CoS benchmarks (PASCAL, iCoseg, Internet and MSRC), three CoSD benchmarks (Cosal2015, CoSOD3k, and CocA) and four VSOD benchmarks (DAVIS16, FBMS, ViSal and SegV2) show that our method outperforms other state-of-the-arts on three different tasks in both accuracy and speed by using the same network architecture , which can reach 140 FPS in real-time.","url_abs":"https://arxiv.org/abs/2203.04708v2","url_pdf":"https://arxiv.org/pdf/2203.04708v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-unified-transformer-framework-for-group","repo_url":"https://github.com/suyukun666/UFO","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"co-saliency-detection","task_name":"Co-Salient Object Detection"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"saliency-detection","task_name":"Saliency Detection"},{"task_slug":"salient-object-detection-1","task_name":"Salient Object Detection"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-salient-object-detection","task_name":"Video Salient Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/co-salient-object-detection-on-coca","task":"Co-Salient Object Detection","dataset":"CoCA","model":"UFO","rank_in_archive_order":4,"of":10,"metrics":{"MAE":"0.095","Mean F-measure":"0.555","S-measure":"0.697","max E-measure":"0.782","max F-measure":"0.571","mean E-measure":"0.762"},"uses_additional_data":false},{"leaderboard":"/sota/co-salient-object-detection-on-cosod3k","task":"Co-Salient Object Detection","dataset":"CoSOD3k","model":"UFO","rank_in_archive_order":3,"of":10,"metrics":{"MAE":"0.073","S-measure":"0.819","max E-measure":"0.874","max F-measure":"0.797","mean E-measure":"0.855","mean F-measure":"0.783"},"uses_additional_data":false},{"leaderboard":"/sota/co-salient-object-detection-on-cosal2015","task":"Co-Salient Object Detection","dataset":"CoSal2015","model":"UFO","rank_in_archive_order":4,"of":10,"metrics":{"MAE":"0.064","S-measure":"0.860","max E-measure":"0.906","max F-measure":"0.865","mean E-measure":"0.889","mean F-measure":"0.848"},"uses_additional_data":false},{"leaderboard":"/sota/co-salient-object-detection-on-icoseg","task":"Co-Salient Object Detection","dataset":"iCoSeg","model":"UFO","rank_in_archive_order":1,"of":2,"metrics":{"MAE":"0.029","S-measure":"0.924","max E-measure":"0.969","max F-measure":"0.953"},"uses_additional_data":false},{"leaderboard":"/sota/video-salient-object-detection-on-davis-2016","task":"Video Salient Object Detection","dataset":"DAVIS-2016","model":"UFO","rank_in_archive_order":2,"of":11,"metrics":{"AVERAGE MAE":"0.015","MAX F-MEASURE":"0.906","S-Measure":"0.918"},"uses_additional_data":true},{"leaderboard":"/sota/video-salient-object-detection-on-fbms-59","task":"Video Salient Object Detection","dataset":"FBMS-59","model":"UFO","rank_in_archive_order":2,"of":16,"metrics":{"AVERAGE MAE":"0.028","MAX F-MEASURE":"0.890","S-Measure":"0.894"},"uses_additional_data":false},{"leaderboard":"/sota/video-salient-object-detection-on-segtrack-v2","task":"Video Salient Object Detection","dataset":"SegTrack v2","model":"UFO","rank_in_archive_order":1,"of":8,"metrics":{"AVERAGE MAE":"0.022","MAX F-MEASURE":"0.863","S-Measure":"0.892"},"uses_additional_data":true},{"leaderboard":"/sota/video-salient-object-detection-on-visal","task":"Video Salient Object Detection","dataset":"ViSal","model":"UFO","rank_in_archive_order":2,"of":10,"metrics":{"Average MAE":"0.011","S-Measure":"0.953","max E-measure":"0.987"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.04708","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.04708"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/suyukun666/UFO","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":13},"by_repo_kind":{"official":{"samples":13,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b17f0cf589616633","entry":"Idx","repo":"suyukun666/UFO","repo_kind":"official","path":"demo_matching.py","file_url":"https://github.com/suyukun666/UFO/blob/HEAD/demo_matching.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b17f0cf589616633"}},{"code_sha256_prefix":"dbdcf302f4b3a47c","entry":"cal_fmeasure","repo":"suyukun666/UFO","repo_kind":"official","path":"misc.py","file_url":"https://github.com/suyukun666/UFO/blob/HEAD/misc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dbdcf302f4b3a47c"}},{"code_sha256_prefix":"d31192c9faa032ba","entry":"cal_precision_recall_mae","repo":"suyukun666/UFO","repo_kind":"official","path":"misc.py","file_url":"https://github.com/suyukun666/UFO/blob/HEAD/misc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d31192c9faa032ba"}},{"code_sha256_prefix":"e0f66060ff14683c","entry":"calc_precision_and_jaccard","repo":"suyukun666/UFO","repo_kind":"official","path":"tools.py","file_url":"https://github.com/suyukun666/UFO/blob/HEAD/tools.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e0f66060ff14683c"}},{"code_sha256_prefix":"f3de7abdbee03ebc","entry":"cls_modulation_branch","repo":"suyukun666/UFO","repo_kind":"official","path":"model_image.py","file_url":"https://github.com/suyukun666/UFO/blob/HEAD/model_image.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f3de7abdbee03ebc"}},{"code_sha256_prefix":"84645a1ce9d31efc","entry":"filt_small_instance","repo":"suyukun666/UFO","repo_kind":"official","path":"data_processed.py","file_url":"https://github.com/suyukun666/UFO/blob/HEAD/data_processed.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"84645a1ce9d31efc"}},{"code_sha256_prefix":"538d5931620a465b","entry":"generate_binary_map","repo":"suyukun666/UFO","repo_kind":"official","path":"tools.py","file_url":"https://github.com/suyukun666/UFO/blob/HEAD/tools.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"538d5931620a465b"}},{"code_sha256_prefix":"a72015580babe1d1","entry":"hsp","repo":"suyukun666/UFO","repo_kind":"official","path":"model_image.py","file_url":"https://github.com/suyukun666/UFO/blob/HEAD/model_image.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a72015580babe1d1"}},{"code_sha256_prefix":"e35f627f91cfd0df","entry":"img_normalize","repo":"suyukun666/UFO","repo_kind":"official","path":"data_processed.py","file_url":"https://github.com/suyukun666/UFO/blob/HEAD/data_processed.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e35f627f91cfd0df"}},{"code_sha256_prefix":"8301a9d5dbc96a5b","entry":"index_points","repo":"suyukun666/UFO","repo_kind":"official","path":"Intra_MLP.py","file_url":"https://github.com/suyukun666/UFO/blob/HEAD/Intra_MLP.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8301a9d5dbc96a5b"}},{"code_sha256_prefix":"4e5ea5f4f33f1db3","entry":"knn_l2","repo":"suyukun666/UFO","repo_kind":"official","path":"Intra_MLP.py","file_url":"https://github.com/suyukun666/UFO/blob/HEAD/Intra_MLP.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4e5ea5f4f33f1db3"}},{"code_sha256_prefix":"b3ea62a04ba82560","entry":"pix_idx","repo":"suyukun666/UFO","repo_kind":"official","path":"demo_matching.py","file_url":"https://github.com/suyukun666/UFO/blob/HEAD/demo_matching.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b3ea62a04ba82560"}},{"code_sha256_prefix":"836db4ac2f48034c","entry":"vgg","repo":"suyukun666/UFO","repo_kind":"official","path":"model_image.py","file_url":"https://github.com/suyukun666/UFO/blob/HEAD/model_image.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"836db4ac2f48034c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}