{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/disentangled-pre-training-for-human-object","title":"Disentangled Pre-training for Human-Object Interaction Detection","arxiv_id":"2404.01725","date":"2024-04-02","proceeding":"CVPR 2024 1","authors":["Zhuolong Li","Xingao Li","Changxing Ding","Xiangmin Xu"],"abstract":"Detecting human-object interaction (HOI) has long been limited by the amount of supervised data available. Recent approaches address this issue by pre-training according to pseudo-labels, which align object regions with HOI triplets parsed from image captions. However, pseudo-labeling is tricky and noisy, making HOI pre-training a complex process. Therefore, we propose an efficient disentangled pre-training method for HOI detection (DP-HOI) to address this problem. First, DP-HOI utilizes object detection and action recognition datasets to pre-train the detection and interaction decoder layers, respectively. Then, we arrange these decoder layers so that the pre-training architecture is consistent with the downstream HOI detection task. This facilitates efficient knowledge transfer. Specifically, the detection decoder identifies reliable human instances in each action recognition dataset image, generates one corresponding query, and feeds it into the interaction decoder for verb classification. Next, we combine the human instance verb predictions in the same image and impose image-level supervision. The DP-HOI structure can be easily adapted to the HOI detection task, enabling effective model parameter initialization. Therefore, it significantly enhances the performance of existing HOI detection models on a broad range of rare categories. The code and pre-trained weight are available at https://github.com/xingaoli/DP-HOI.","url_abs":"https://arxiv.org/abs/2404.01725v1","url_pdf":"https://arxiv.org/pdf/2404.01725v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"disentangled-pre-training-for-human-object","repo_url":"https://github.com/xingaoli/dp-hoi","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"human-object-interaction-detection","task_name":"Human-Object Interaction Detection"},{"task_slug":"image-captioning","task_name":"Image Captioning"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.01725","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.01725"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/xingaoli/DP-HOI","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_violates":1,"ran_fixture":1,"ran":2,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ac8fe530cdad4d8c","entry":"dice_loss","repo":"xingaoli/DP-HOI","repo_kind":"official","path":"models/segmentation.py","file_url":"https://github.com/xingaoli/DP-HOI/blob/HEAD/models/segmentation.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ac8fe530cdad4d8c"}},{"code_sha256_prefix":"9db6a2fdc3039d38","entry":"dn_post_process","repo":"xingaoli/DP-HOI","repo_kind":"official","path":"models/dn_components.py","file_url":"https://github.com/xingaoli/DP-HOI/blob/HEAD/models/dn_components.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9db6a2fdc3039d38"}},{"code_sha256_prefix":"47f1cef8edcb4695","entry":"prepare_for_loss","repo":"xingaoli/DP-HOI","repo_kind":"official","path":"models/dn_components.py","file_url":"https://github.com/xingaoli/DP-HOI/blob/HEAD/models/dn_components.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"47f1cef8edcb4695"}},{"code_sha256_prefix":"ef152cffea106bb8","entry":"process_video_to_frames","repo":"xingaoli/DP-HOI","repo_kind":"official","path":"pre_datasets/pre_kinetics700.py","file_url":"https://github.com/xingaoli/DP-HOI/blob/HEAD/pre_datasets/pre_kinetics700.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ef152cffea106bb8"}},{"code_sha256_prefix":"5c0711aada67957e","entry":"sigmoid_focal_loss","repo":"xingaoli/DP-HOI","repo_kind":"official","path":"models/segmentation.py","file_url":"https://github.com/xingaoli/DP-HOI/blob/HEAD/models/segmentation.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5c0711aada67957e"}},{"code_sha256_prefix":"fdc5e45115a1f362","entry":"build_transformer","repo":"xingaoli/DP-HOI","repo_kind":"official","path":"models/transformer.py","file_url":"https://github.com/xingaoli/DP-HOI/blob/HEAD/models/transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fdc5e45115a1f362"}},{"code_sha256_prefix":"06f8c01bf94e4524","entry":"prepare_for_dn","repo":"xingaoli/DP-HOI","repo_kind":"official","path":"models/dn_components.py","file_url":"https://github.com/xingaoli/DP-HOI/blob/HEAD/models/dn_components.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"06f8c01bf94e4524"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}