{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2508-03207","title":"Open-Vocabulary HOI Detection with Interaction-aware Prompt and Concept Calibration","arxiv_id":"2508.03207","date":"2025-08-05","proceeding":"ICCV","authors":["Ting Lei","Shaofeng Yin","Qingchao Chen","Yuxin Peng","Yang Liu"],"abstract":"Open Vocabulary Human-Object Interaction (HOI) detection aims to detect interactions between humans and objects while generalizing to novel interaction classes beyond the training set. Current methods often rely on Vision and Language Models (VLMs) but face challenges due to suboptimal image encoders, as image-level pre-training does not align well with the fine-grained region-level interaction detection required for HOI. Additionally, effectively encoding textual descriptions of visual appearances remains difficult, limiting the model's ability to capture detailed HOI relationships. To address these issues, we propose INteraction-aware Prompting with Concept Calibration (INP-CC), an end-to-end open-vocabulary HOI detector that integrates interaction-aware prompts and concept calibration. Specifically, we propose an interaction-aware prompt generator that dynamically generates a compact set of prompts based on the input scene, enabling selective sharing among similar interactions. This approach directs the model's attention to key interaction patterns rather than generic image-level semantics, enhancing HOI detection. Furthermore, we refine HOI concept representations through language model-guided calibration, which helps distinguish diverse HOI concepts by investigating visual similarities across categories. A negative sampling strategy is also employed to improve inter-modal similarity modeling, enabling the model to better differentiate visually similar but semantically distinct actions. Extensive experimental results demonstrate that INP-CC significantly outperforms state-of-the-art models on the SWIG-HOI and HICO-DET datasets. Code is available at https://github.com/ltttpku/INP-CC.","url_abs":"https://arxiv.org/abs/2508.03207","url_pdf":"https://arxiv.org/pdf/2508.03207","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2508.03207","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2508.03207"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/ltttpku/INP-CC","reach":null}],"summary":{"ran":2,"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"found_in_text":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"a39d592c3fa84805","entry":"HOIResidualAttentionBlock","repo":"ltttpku/INP-CC","repo_kind":"found_in_text","path":"models/model.py","file_url":"https://github.com/ltttpku/INP-CC/blob/HEAD/models/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a39d592c3fa84805"}},{"code_sha256_prefix":"99315d730e76026b","entry":"TransformerDecoder","repo":"ltttpku/INP-CC","repo_kind":"found_in_text","path":"models/model.py","file_url":"https://github.com/ltttpku/INP-CC/blob/HEAD/models/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"99315d730e76026b"}},{"code_sha256_prefix":"3ae02165f1654790","entry":"_get_clones","repo":"ltttpku/INP-CC","repo_kind":"found_in_text","path":"models/model.py","file_url":"https://github.com/ltttpku/INP-CC/blob/HEAD/models/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3ae02165f1654790"}},{"code_sha256_prefix":"51dec3db98d46348","entry":"HOITransformer","repo":"ltttpku/INP-CC","repo_kind":"found_in_text","path":"models/model.py","file_url":"https://github.com/ltttpku/INP-CC/blob/HEAD/models/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"51dec3db98d46348"}},{"code_sha256_prefix":"8f2b5ad72275f89c","entry":"HOIVisionTransformer","repo":"ltttpku/INP-CC","repo_kind":"found_in_text","path":"models/model.py","file_url":"https://github.com/ltttpku/INP-CC/blob/HEAD/models/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8f2b5ad72275f89c"}},{"code_sha256_prefix":"618361e80bbd4d42","entry":"TransformerDecoderLayer","repo":"ltttpku/INP-CC","repo_kind":"found_in_text","path":"models/model.py","file_url":"https://github.com/ltttpku/INP-CC/blob/HEAD/models/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"618361e80bbd4d42"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_api"},"syntology_extracted_results":null}