{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-two-stage-detection-of-human-object","title":"Efficient Two-Stage Detection of Human-Object Interactions with a Novel Unary-Pairwise Transformer","arxiv_id":"2112.01838","date":"2021-12-03","proceeding":"CVPR 2022 1","authors":["Frederic Z. Zhang","Dylan Campbell","Stephen Gould"],"abstract":"Recent developments in transformer models for visual data have led to significant improvements in recognition and detection tasks. In particular, using learnable queries in place of region proposals has given rise to a new class of one-stage detection models, spearheaded by the Detection Transformer (DETR). Variations on this one-stage approach have since dominated human-object interaction (HOI) detection. However, the success of such one-stage HOI detectors can largely be attributed to the representation power of transformers. We discovered that when equipped with the same transformer, their two-stage counterparts can be more performant and memory-efficient, while taking a fraction of the time to train. In this work, we propose the Unary-Pairwise Transformer, a two-stage detector that exploits unary and pairwise representations for HOIs. We observe that the unary and pairwise parts of our transformer network specialise, with the former preferentially increasing the scores of positive examples and the latter decreasing the scores of negative examples. We evaluate our method on the HICO-DET and V-COCO datasets, and significantly outperform state-of-the-art approaches. At inference time, our model with ResNet50 approaches real-time performance on a single GPU.","url_abs":"https://arxiv.org/abs/2112.01838v2","url_pdf":"https://arxiv.org/pdf/2112.01838v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-two-stage-detection-of-human-object","repo_url":"https://github.com/fredzzhang/upt","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[{"task_slug":null,"task_name":"GPU"},{"task_slug":"human-object-interaction-detection","task_name":"Human-Object Interaction Detection"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/human-object-interaction-detection-on-hico","task":"Human-Object Interaction Detection","dataset":"HICO-DET","model":"UPT-R101-DC5","rank_in_archive_order":20,"of":55,"metrics":{"Time Per Frame (ms)":"124","mAP":"32.62"},"uses_additional_data":false},{"leaderboard":"/sota/human-object-interaction-detection-on-hico","task":"Human-Object Interaction Detection","dataset":"HICO-DET","model":"UPT-R101","rank_in_archive_order":22,"of":55,"metrics":{"Time Per Frame (ms)":"61","mAP":"32.31"},"uses_additional_data":false},{"leaderboard":"/sota/human-object-interaction-detection-on-hico","task":"Human-Object Interaction Detection","dataset":"HICO-DET","model":"UPT-R50","rank_in_archive_order":25,"of":55,"metrics":{"Time Per Frame (ms)":"42","mAP":"31.66"},"uses_additional_data":false},{"leaderboard":"/sota/human-object-interaction-detection-on-v-coco","task":"Human-Object Interaction Detection","dataset":"V-COCO","model":"UPT-R101-DC5","rank_in_archive_order":10,"of":34,"metrics":{"AP(S1)":"61.3","AP(S2)":"67.1","Time Per Frame(ms)":"131"},"uses_additional_data":false},{"leaderboard":"/sota/human-object-interaction-detection-on-v-coco","task":"Human-Object Interaction Detection","dataset":"V-COCO","model":"UPT-R101","rank_in_archive_order":11,"of":34,"metrics":{"AP(S1)":"60.7","AP(S2)":"66.2","Time Per Frame(ms)":"64"},"uses_additional_data":false},{"leaderboard":"/sota/human-object-interaction-detection-on-v-coco","task":"Human-Object Interaction Detection","dataset":"V-COCO","model":"UPT-R50","rank_in_archive_order":12,"of":34,"metrics":{"AP(S1)":"59.0","AP(S2)":"64.5","Time Per Frame(ms)":"43"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2112.01838","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.01838"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/fredzzhang/upt","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"summary":{"ran_honours":2,"unverified":2},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e0a06ded5d4f6c3c","entry":"box_cxcywh_to_xyxy","repo":"fredzzhang/upt","repo_kind":"official","path":"ops.py","file_url":"https://github.com/fredzzhang/upt/blob/HEAD/ops.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"e0a06ded5d4f6c3c"}},{"code_sha256_prefix":"031b02fb990284ba","entry":"box_xyxy_to_cxcywh","repo":"fredzzhang/upt","repo_kind":"official","path":"ops.py","file_url":"https://github.com/fredzzhang/upt/blob/HEAD/ops.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"031b02fb990284ba"}},{"code_sha256_prefix":"000414180baf0027","entry":"compute_spatial_encodings","repo":"fredzzhang/upt","repo_kind":"official","path":"ops.py","file_url":"https://github.com/fredzzhang/upt/blob/HEAD/ops.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"000414180baf0027"}},{"code_sha256_prefix":"8c73b8b7646a5ae4","entry":"custom_collate","repo":"fredzzhang/upt","repo_kind":"official","path":"utils.py","file_url":"https://github.com/fredzzhang/upt/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"8c73b8b7646a5ae4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}