{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/relationnet-bridging-visual-representations","title":"RelationNet++: Bridging Visual Representations for Object Detection via Transformer Decoder","arxiv_id":"2010.15831","date":"2020-10-29","proceeding":"NeurIPS 2020 12","authors":["Cheng Chi","Fangyun Wei","Han Hu"],"abstract":"Existing object detection frameworks are usually built on a single format of object/part representation, i.e., anchor/proposal rectangle boxes in RetinaNet and Faster R-CNN, center points in FCOS and RepPoints, and corner points in CornerNet. While these different representations usually drive the frameworks to perform well in different aspects, e.g., better classification or finer localization, it is in general difficult to combine these representations in a single framework to make good use of each strength, due to the heterogeneous or non-grid feature extraction by different representations. This paper presents an attention-based decoder module similar as that in Transformer~\\cite{vaswani2017attention} to bridge other representations into a typical object detector built on a single representation format, in an end-to-end fashion. The other representations act as a set of \\emph{key} instances to strengthen the main \\emph{query} representation features in the vanilla detectors. Novel techniques are proposed towards efficient computation of the decoder module, including a \\emph{key sampling} approach and a \\emph{shared location embedding} approach. The proposed module is named \\emph{bridging visual representations} (BVR). It can perform in-place and we demonstrate its broad effectiveness in bridging other representations into prevalent object detection frameworks, including RetinaNet, Faster R-CNN, FCOS and ATSS, where about $1.5\\sim3.0$ AP improvements are achieved. In particular, we improve a state-of-the-art framework with a strong backbone by about $2.0$ AP, reaching $52.7$ AP on COCO test-dev. The resulting network is named RelationNet++. The code will be available at https://github.com/microsoft/RelationNet2.","url_abs":"https://arxiv.org/abs/2010.15831v1","url_pdf":"https://arxiv.org/pdf/2010.15831v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"relationnet-bridging-visual-representations","repo_url":"https://github.com/microsoft/RelationNet2","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"relationnet-bridging-visual-representations","repo_url":"https://github.com/MindSpore-paper-code-2/code2/tree/main/relationnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"relationnet-bridging-visual-representations","repo_url":"https://github.com/MindSpore-paper-code-3/code5/tree/main/relationnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"relationnet-bridging-visual-representations","repo_url":"https://github.com/shinya7y/UniverseNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"atss","method_name":"ATSS"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"corner-pooling","method_name":"Corner Pooling"},{"method_slug":"cornernet","method_name":"CornerNet"},{"method_slug":"fcos","method_name":"FCOS"},{"method_slug":"fpn","method_name":"FPN"},{"method_slug":"faster-r-cnn","method_name":"Faster R-CNN"},{"method_slug":"focal-loss","method_name":"Focal Loss"},{"method_slug":"hourglass-module","method_name":"Hourglass Module"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"non-maximum-suppression","method_name":"Non Maximum Suppression"},{"method_slug":"rpn","method_name":"RPN"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"reppoints","method_name":"RepPoints"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"retinanet","method_name":"RetinaNet"},{"method_slug":"roipool","method_name":"RoIPool"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"stacked-hourglass-network","method_name":"Stacked Hourglass Network"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/object-detection-on-coco","task":"Object Detection","dataset":"COCO test-dev","model":"RelationNet++ (ResNeXt-64x4d-101-DCN)","rank_in_archive_order":68,"of":225,"metrics":{"box mAP":"52.7"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2010.15831","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2010.15831"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindSpore-paper-code-2/code2/tree/main/relationnet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindSpore-paper-code-3/code5/tree/main/relationnet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/microsoft/RelationNet2","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shinya7y/UniverseNet","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":5},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"69d11c2f1a409fff","entry":"anchorfree_forward_feature_single","repo":"microsoft/RelationNet2","repo_kind":"official","path":"code/models/utils/bvr_utils.py","file_url":"https://github.com/microsoft/RelationNet2/blob/HEAD/code/models/utils/bvr_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"69d11c2f1a409fff"}},{"code_sha256_prefix":"432371ded78e0429","entry":"atss_forward_prediction_single","repo":"microsoft/RelationNet2","repo_kind":"official","path":"code/models/utils/bvr_utils.py","file_url":"https://github.com/microsoft/RelationNet2/blob/HEAD/code/models/utils/bvr_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"432371ded78e0429"}},{"code_sha256_prefix":"fd2a3221f3c55b7f","entry":"gaussian_radius","repo":"microsoft/RelationNet2","repo_kind":"official","path":"code/core/bbox/assigners/point_kpt_assigner.py","file_url":"https://github.com/microsoft/RelationNet2/blob/HEAD/code/core/bbox/assigners/point_kpt_assigner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fd2a3221f3c55b7f"}},{"code_sha256_prefix":"150c213138c1ac1a","entry":"multi_head_attention_forward","repo":"microsoft/RelationNet2","repo_kind":"official","path":"code/models/utils/bvr_transformer/multihead_attention.py","file_url":"https://github.com/microsoft/RelationNet2/blob/HEAD/code/models/utils/bvr_transformer/multihead_attention.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"150c213138c1ac1a"}},{"code_sha256_prefix":"57ba5c957e237b5f","entry":"reduce_mean","repo":"microsoft/RelationNet2","repo_kind":"official","path":"code/models/dense_heads/keypoint_head.py","file_url":"https://github.com/microsoft/RelationNet2/blob/HEAD/code/models/dense_heads/keypoint_head.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"57ba5c957e237b5f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}