{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/seggpt-segmenting-everything-in-context","title":"SegGPT: Segmenting Everything In Context","arxiv_id":"2304.03284","date":"2023-04-06","proceeding":null,"authors":["Xinlong Wang","Xiaosong Zhang","Yue Cao","Wen Wang","Chunhua Shen","Tiejun Huang"],"abstract":"We present SegGPT, a generalist model for segmenting everything in context. We unify various segmentation tasks into a generalist in-context learning framework that accommodates different kinds of segmentation data by transforming them into the same format of images. The training of SegGPT is formulated as an in-context coloring problem with random color mapping for each data sample. The objective is to accomplish diverse tasks according to the context, rather than relying on specific colors. After training, SegGPT can perform arbitrary segmentation tasks in images or videos via in-context inference, such as object instance, stuff, part, contour, and text. SegGPT is evaluated on a broad range of tasks, including few-shot semantic segmentation, video object segmentation, semantic segmentation, and panoptic segmentation. Our results show strong capabilities in segmenting in-domain and out-of-domain targets, either qualitatively or quantitatively.","url_abs":"https://arxiv.org/abs/2304.03284v1","url_pdf":"https://arxiv.org/pdf/2304.03284v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"seggpt-segmenting-everything-in-context","repo_url":"https://github.com/baaivision/painter","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"seggpt-segmenting-everything-in-context","repo_url":"https://github.com/pwc-1/Paper-10/tree/main/seggpt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"seggpt-segmenting-everything-in-context","repo_url":"https://github.com/yangyucheng000/University/tree/main/model-3/seggpt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"few-shot-image-segmentation","task_name":"Few-Shot Semantic Segmentation"},{"task_slug":"in-context-learning","task_name":"In-Context Learning"},{"task_slug":"panoptic-segmentation","task_name":"Panoptic Segmentation"},{"task_slug":"personalized-segmentation","task_name":"Personalized Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-object-segmentation","task_name":"Video Object Segmentation"},{"task_slug":"video-semantic-segmentation","task_name":"Video Semantic Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/few-shot-semantic-segmentation-on-coco-20i-1","task":"Few-Shot Semantic Segmentation","dataset":"COCO-20i (1-shot)","model":"SegGPT (ViT)","rank_in_archive_order":3,"of":85,"metrics":{"Mean IoU":"56.1"},"uses_additional_data":true},{"leaderboard":"/sota/few-shot-semantic-segmentation-on-coco-20i-5","task":"Few-Shot Semantic Segmentation","dataset":"COCO-20i (5-shot)","model":"SegGPT (ViT)","rank_in_archive_order":1,"of":81,"metrics":{"Mean IoU":"67.9"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-semantic-segmentation-on-fss-1000-1","task":"Few-Shot Semantic Segmentation","dataset":"FSS-1000 (1-shot)","model":"SegGPT (ViT)","rank_in_archive_order":20,"of":24,"metrics":{"Mean IoU":"85.6"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-semantic-segmentation-on-fss-1000-5","task":"Few-Shot Semantic Segmentation","dataset":"FSS-1000 (5-shot)","model":"SegGPT (ViT)","rank_in_archive_order":9,"of":22,"metrics":{"Mean IoU":"89.3"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-semantic-segmentation-on-pascal-5i-1","task":"Few-Shot Semantic Segmentation","dataset":"PASCAL-5i (1-Shot)","model":"SegGPT (ViT)","rank_in_archive_order":1,"of":105,"metrics":{"Mean IoU":"83.2"},"uses_additional_data":true},{"leaderboard":"/sota/few-shot-semantic-segmentation-on-pascal-5i-5","task":"Few-Shot Semantic Segmentation","dataset":"PASCAL-5i (5-Shot)","model":"SegGPT (ViT)","rank_in_archive_order":1,"of":96,"metrics":{"Mean IoU":"89.8"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2304.03284","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.03284"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yangyucheng000/University/tree/main/model-3/seggpt","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/baaivision/painter","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pwc-1/Paper-10/tree/main/seggpt","reach":null}],"summary":{"ran_honours":2,"ran":4,"unverified":4},"by_repo_kind":{"official":{"samples":10,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bd3578806ecf5ac2","entry":"adjust_learning_rate","repo":"baaivision/painter","repo_kind":"official","path":"Painter/util/lr_sched.py","file_url":"https://github.com/baaivision/painter/blob/HEAD/Painter/util/lr_sched.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bd3578806ecf5ac2"}},{"code_sha256_prefix":"27f8c4c706fd43ec","entry":"build_dataset","repo":"baaivision/painter","repo_kind":"official","path":"Painter/util/datasets.py","file_url":"https://github.com/baaivision/painter/blob/HEAD/Painter/util/datasets.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"27f8c4c706fd43ec"}},{"code_sha256_prefix":"800cebf161a8aa26","entry":"build_transform","repo":"baaivision/painter","repo_kind":"official","path":"Painter/util/datasets.py","file_url":"https://github.com/baaivision/painter/blob/HEAD/Painter/util/datasets.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"800cebf161a8aa26"}},{"code_sha256_prefix":"f55c31d782e48ffb","entry":"collate_fn","repo":"baaivision/painter","repo_kind":"official","path":"Painter/util/ddp_utils.py","file_url":"https://github.com/baaivision/painter/blob/HEAD/Painter/util/ddp_utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f55c31d782e48ffb"}},{"code_sha256_prefix":"89edf5372214b57d","entry":"get_layer_id_for_vit","repo":"baaivision/painter","repo_kind":"official","path":"Painter/util/lr_decay.py","file_url":"https://github.com/baaivision/painter/blob/HEAD/Painter/util/lr_decay.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"89edf5372214b57d"}},{"code_sha256_prefix":"8baf25cef41cba16","entry":"param_groups_lrd","repo":"baaivision/painter","repo_kind":"official","path":"Painter/util/lr_decay.py","file_url":"https://github.com/baaivision/painter/blob/HEAD/Painter/util/lr_decay.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8baf25cef41cba16"}},{"code_sha256_prefix":"0662163cbe22b24d","entry":"get_loss_scale_for_deepspeed","repo":"baaivision/painter","repo_kind":"official","path":"Painter/engine_train.py","file_url":"https://github.com/baaivision/painter/blob/HEAD/Painter/engine_train.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0662163cbe22b24d"}},{"code_sha256_prefix":"7fd0e9f741ef35d4","entry":"get_vit_lr_decay_rate","repo":"baaivision/painter","repo_kind":"official","path":"Painter/models_painter.py","file_url":"https://github.com/baaivision/painter/blob/HEAD/Painter/models_painter.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7fd0e9f741ef35d4"}},{"code_sha256_prefix":"b21729609ddb1591","entry":"init_distributed_mode","repo":"baaivision/painter","repo_kind":"official","path":"Painter/util/ddp_utils.py","file_url":"https://github.com/baaivision/painter/blob/HEAD/Painter/util/ddp_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b21729609ddb1591"}},{"code_sha256_prefix":"9f23def85e2c60a8","entry":"mask_matrix_nms","repo":"baaivision/painter","repo_kind":"official","path":"Painter/util/matrix_nms.py","file_url":"https://github.com/baaivision/painter/blob/HEAD/Painter/util/matrix_nms.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f23def85e2c60a8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}