{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/object-contextual-representations-for","title":"Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation","arxiv_id":"1909.11065","date":"2019-09-24","proceeding":"ECCV 2020 8","authors":["Yuhui Yuan","Xiaokang Chen","Xilin Chen","Jingdong Wang"],"abstract":"In this paper, we address the semantic segmentation problem with a focus on the context aggregation strategy. Our motivation is that the label of a pixel is the category of the object that the pixel belongs to. We present a simple yet effective approach, object-contextual representations, characterizing a pixel by exploiting the representation of the corresponding object class. First, we learn object regions under the supervision of ground-truth segmentation. Second, we compute the object region representation by aggregating the representations of the pixels lying in the object region. Last, % the representation similarity we compute the relation between each pixel and each object region and augment the representation of each pixel with the object-contextual representation which is a weighted aggregation of all the object region representations according to their relations with the pixel. We empirically demonstrate that the proposed approach achieves competitive performance on various challenging semantic segmentation benchmarks: Cityscapes, ADE20K, LIP, PASCAL-Context, and COCO-Stuff. Cityscapes, ADE20K, LIP, PASCAL-Context, and COCO-Stuff. Our submission \"HRNet + OCR + SegFix\" achieves 1-st place on the Cityscapes leaderboard by the time of submission. Code is available at: https://git.io/openseg and https://git.io/HRNet.OCR. We rephrase the object-contextual representation scheme using the Transformer encoder-decoder framework. The details are presented in~Section3.3.","url_abs":"https://arxiv.org/abs/1909.11065v6","url_pdf":"https://arxiv.org/pdf/1909.11065v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"object-contextual-representations-for","repo_url":"https://github.com/HRNet/HRNet-Semantic-Segmentation","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"object-contextual-representations-for","repo_url":"https://github.com/Burf/HRNetV2-OCR-Tensorflow2","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"object-contextual-representations-for","repo_url":"https://github.com/JanMarcelKezmann/TensorFlow-Advanced-Segmentation-Models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"object-contextual-representations-for","repo_url":"https://github.com/openseg-group/openseg.pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"object-contextual-representations-for","repo_url":"https://github.com/rosinality/ocr-pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"object-contextual-representations-for","repo_url":"https://github.com/2024-MindSpore-1/Code4/tree/main/OCRNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"object-contextual-representations-for","repo_url":"https://github.com/MindSpore-paper-code-3/code9/tree/main/OCRNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"object-contextual-representations-for","repo_url":"https://github.com/PaddlePaddle/PaddleSeg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"paddle","reach":null},{"paper_slug":"object-contextual-representations-for","repo_url":"https://github.com/kingcong/OCRNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}},{"paper_slug":"object-contextual-representations-for","repo_url":"https://github.com/mindspore-ai/models/tree/master/research/cv/OCRNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"object-contextual-representations-for","repo_url":"https://github.com/open-mmlab/mmsegmentation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"object","task_name":"Object"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/semantic-segmentation-on-ade20k","task":"Semantic Segmentation","dataset":"ADE20K","model":"HRNetV2 + OCR + RMI (PaddleClas pretrained)","rank_in_archive_order":155,"of":235,"metrics":{"Validation mIoU":"47.98"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-ade20k","task":"Semantic Segmentation","dataset":"ADE20K","model":"OCR(HRNetV2-W48)","rank_in_archive_order":187,"of":235,"metrics":{"Validation mIoU":"45.66"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-ade20k","task":"Semantic Segmentation","dataset":"ADE20K","model":"OCR (ResNet-101)","rank_in_archive_order":193,"of":235,"metrics":{"Validation mIoU":"45.28"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-ade20k-val","task":"Semantic Segmentation","dataset":"ADE20K val","model":"HRNetV2 + OCR + RMI (PaddleClas pretrained)","rank_in_archive_order":63,"of":95,"metrics":{"mIoU":"47.98"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-ade20k-val","task":"Semantic Segmentation","dataset":"ADE20K val","model":"OCR (HRNetV2-W48)","rank_in_archive_order":79,"of":95,"metrics":{"mIoU":"45.66"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-ade20k-val","task":"Semantic Segmentation","dataset":"ADE20K val","model":"OCR (ResNet-101)","rank_in_archive_order":81,"of":95,"metrics":{"mIoU":"45.28"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-bdd100k-val","task":"Semantic Segmentation","dataset":"BDD100K val","model":"OCRNet","rank_in_archive_order":8,"of":24,"metrics":{"mIoU":"60.1"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-coco-stuff-test","task":"Semantic Segmentation","dataset":"COCO-Stuff test","model":"HRNetV2 + OCR + RMI (PaddleClas pretrained)","rank_in_archive_order":9,"of":21,"metrics":{"mIoU":"45.2%"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-coco-stuff-test","task":"Semantic Segmentation","dataset":"COCO-Stuff test","model":"OCR (HRNetV2-W48)","rank_in_archive_order":12,"of":21,"metrics":{"mIoU":"40.5%"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-coco-stuff-test","task":"Semantic Segmentation","dataset":"COCO-Stuff test","model":"OCR (ResNet-101)","rank_in_archive_order":16,"of":21,"metrics":{"mIoU":"39.5%"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-cityscapes","task":"Semantic Segmentation","dataset":"Cityscapes test","model":"HRNetV2 + OCR +","rank_in_archive_order":9,"of":105,"metrics":{"Mean IoU (class)":"84.5%"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-cityscapes","task":"Semantic Segmentation","dataset":"Cityscapes test","model":"HRNetV2 + OCR (w/ ASP)","rank_in_archive_order":12,"of":105,"metrics":{"Mean IoU (class)":"83.7%"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-cityscapes","task":"Semantic Segmentation","dataset":"Cityscapes test","model":"OCR (HRNetV2-W48, coarse)","rank_in_archive_order":20,"of":105,"metrics":{"Mean IoU (class)":"83.0%"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-cityscapes","task":"Semantic Segmentation","dataset":"Cityscapes test","model":"OCR (ResNet-101, coarse)","rank_in_archive_order":27,"of":105,"metrics":{"Mean IoU (class)":"82.4%"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-cityscapes","task":"Semantic Segmentation","dataset":"Cityscapes test","model":"OCR (ResNet-101)","rank_in_archive_order":34,"of":105,"metrics":{"Mean IoU (class)":"81.8%"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-cityscapes-val","task":"Semantic Segmentation","dataset":"Cityscapes val","model":"HRNetV2 + OCR + RMI (PaddleClas pretrained)","rank_in_archive_order":23,"of":99,"metrics":{"mIoU":"83.6"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-cityscapes-val","task":"Semantic Segmentation","dataset":"Cityscapes val","model":"OCR (ResNet-101-FCN)","rank_in_archive_order":48,"of":99,"metrics":{"mIoU":"80.6"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-lip-val","task":"Semantic Segmentation","dataset":"LIP val","model":"HRNetV2 + OCR + RMI (PaddleClas pretrained)","rank_in_archive_order":5,"of":13,"metrics":{"mIoU":"58.2%"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-lip-val","task":"Semantic Segmentation","dataset":"LIP val","model":"OCR (HRNetV2-W48)","rank_in_archive_order":6,"of":13,"metrics":{"mIoU":"56.65%"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-lip-val","task":"Semantic Segmentation","dataset":"LIP val","model":"OCR (ResNet-101)","rank_in_archive_order":8,"of":13,"metrics":{"mIoU":"55.6%"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-pascal-context","task":"Semantic Segmentation","dataset":"PASCAL Context","model":"HRNetV2 + OCR + RMI (PaddleClas pretrained)","rank_in_archive_order":15,"of":66,"metrics":{"mIoU":"59.6"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-pascal-context","task":"Semantic Segmentation","dataset":"PASCAL Context","model":"OCR (HRNetV2-W48)","rank_in_archive_order":24,"of":66,"metrics":{"mIoU":"56.2"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-pascal-context","task":"Semantic Segmentation","dataset":"PASCAL Context","model":"OCR (ResNet-101)","rank_in_archive_order":33,"of":66,"metrics":{"mIoU":"54.8"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-pascal-voc-2012","task":"Semantic Segmentation","dataset":"PASCAL VOC 2012 test","model":"OCR (HRNetV2-W48)","rank_in_archive_order":12,"of":51,"metrics":{"Mean IoU":"84.5%"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-pascal-voc-2012","task":"Semantic Segmentation","dataset":"PASCAL VOC 2012 test","model":"OCR (ResNet-101)","rank_in_archive_order":13,"of":51,"metrics":{"Mean IoU":"84.3%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1909.11065","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1909.11065"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/PaddlePaddle/PaddleSeg","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mindspore-ai/models/tree/master/research/cv/OCRNet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/2024-MindSpore-1/Code4/tree/main/OCRNet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindSpore-paper-code-3/code9/tree/main/OCRNet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/openseg-group/openseg.pytorch","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Burf/HRNetV2-OCR-Tensorflow2","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/HRNet/HRNet-Semantic-Segmentation","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/open-mmlab/mmsegmentation","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rosinality/ocr-pytorch","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/JanMarcelKezmann/TensorFlow-Advanced-Segmentation-Models","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kingcong/OCRNet","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}}],"summary":{"ran_draft_wrong":4,"unverified":5},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1},"listed":{"samples":8,"ran":3,"repositories":3}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"ab8dda773f3d5789","entry":"conv1d","repo":"rosinality/ocr-pytorch","repo_kind":"listed","path":"model.py","file_url":"https://github.com/rosinality/ocr-pytorch/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ab8dda773f3d5789"}},{"code_sha256_prefix":"9333dd68e11e060d","entry":"conv2d","repo":"rosinality/ocr-pytorch","repo_kind":"listed","path":"model.py","file_url":"https://github.com/rosinality/ocr-pytorch/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9333dd68e11e060d"}},{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"HRNet/HRNet-Semantic-Segmentation","repo_kind":"official","path":"lib/models/seg_hrnet.py","file_url":"https://github.com/HRNet/HRNet-Semantic-Segmentation/blob/HEAD/lib/models/seg_hrnet.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"7d4554a0bd6eb4d2","entry":"generateInstanceStats","repo":"openseg-group/openseg.pytorch","repo_kind":"listed","path":"lib/metrics/cityscapes/evaluation/evalPixelLevelSemanticLabeling.py","file_url":"https://github.com/openseg-group/openseg.pytorch/blob/HEAD/lib/metrics/cityscapes/evaluation/evalPixelLevelSemanticLabeling.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7d4554a0bd6eb4d2"}},{"code_sha256_prefix":"83f120a07b1755b4","entry":"generateMatrix","repo":"openseg-group/openseg.pytorch","repo_kind":"listed","path":"lib/metrics/cityscapes/evaluation/evalPixelLevelSemanticLabeling.py","file_url":"https://github.com/openseg-group/openseg.pytorch/blob/HEAD/lib/metrics/cityscapes/evaluation/evalPixelLevelSemanticLabeling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"83f120a07b1755b4"}},{"code_sha256_prefix":"a9c50973bbd521dc","entry":"getPrediction","repo":"openseg-group/openseg.pytorch","repo_kind":"listed","path":"lib/metrics/cityscapes/evaluation/evalPixelLevelSemanticLabeling.py","file_url":"https://github.com/openseg-group/openseg.pytorch/blob/HEAD/lib/metrics/cityscapes/evaluation/evalPixelLevelSemanticLabeling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a9c50973bbd521dc"}},{"code_sha256_prefix":"6b491db5a0440458","entry":"hr_basic_block","repo":"Burf/HRNetV2-OCR-Tensorflow2","repo_kind":"listed","path":"hrnet/hrnet.py","file_url":"https://github.com/Burf/HRNetV2-OCR-Tensorflow2/blob/HEAD/hrnet/hrnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b491db5a0440458"}},{"code_sha256_prefix":"be1dcef7d7100993","entry":"hr_bottleneck_block","repo":"Burf/HRNetV2-OCR-Tensorflow2","repo_kind":"listed","path":"hrnet/hrnet.py","file_url":"https://github.com/Burf/HRNetV2-OCR-Tensorflow2/blob/HEAD/hrnet/hrnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"be1dcef7d7100993"}},{"code_sha256_prefix":"05831920f7a920c9","entry":"hr_transition_block","repo":"Burf/HRNetV2-OCR-Tensorflow2","repo_kind":"listed","path":"hrnet/hrnet.py","file_url":"https://github.com/Burf/HRNetV2-OCR-Tensorflow2/blob/HEAD/hrnet/hrnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"05831920f7a920c9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}