{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/grounding-3d-object-affordance-from-2d","title":"Grounding 3D Object Affordance from 2D Interactions in Images","arxiv_id":"2303.10437","date":"2023-03-18","proceeding":"ICCV 2023 1","authors":["Yuhang Yang","Wei Zhai","Hongchen Luo","Yang Cao","Jiebo Luo","Zheng-Jun Zha"],"abstract":"Grounding 3D object affordance seeks to locate objects' ''action possibilities'' regions in the 3D space, which serves as a link between perception and operation for embodied agents. Existing studies primarily focus on connecting visual affordances with geometry structures, e.g. relying on annotations to declare interactive regions of interest on the object and establishing a mapping between the regions and affordances. However, the essence of learning object affordance is to understand how to use it, and the manner that detaches interactions is limited in generalization. Normally, humans possess the ability to perceive object affordances in the physical world through demonstration images or videos. Motivated by this, we introduce a novel task setting: grounding 3D object affordance from 2D interactions in images, which faces the challenge of anticipating affordance through interactions of different sources. To address this problem, we devise a novel Interaction-driven 3D Affordance Grounding Network (IAG), which aligns the region feature of objects from different sources and models the interactive contexts for 3D object affordance grounding. Besides, we collect a Point-Image Affordance Dataset (PIAD) to support the proposed task. Comprehensive experiments on PIAD demonstrate the reliability of the proposed task and the superiority of our method. The project is available at https://github.com/yyvhang/IAGNet.","url_abs":"https://arxiv.org/abs/2303.10437v2","url_pdf":"https://arxiv.org/pdf/2303.10437v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"grounding-3d-object-affordance-from-2d","repo_url":"https://github.com/yyvhang/iagnet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"object","task_name":"Object"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.10437","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.10437"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yyvhang/iagnet","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/yyvhang/IAGNet","reach":{"status":"ok"}}],"summary":{"ran":4,"ran_fixture":2,"unverified":6},"by_repo_kind":{"official":{"samples":12,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"21ad4ecf6758102d","entry":"Img_Encoder","repo":"yyvhang/IAGNet","repo_kind":"official","path":"model/IAGNet.py","file_url":"https://github.com/yyvhang/IAGNet/blob/HEAD/model/IAGNet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"21ad4ecf6758102d"}},{"code_sha256_prefix":"8901b6b5925b9d1e","entry":"Inherent_relation","repo":"yyvhang/IAGNet","repo_kind":"official","path":"model/IAGNet.py","file_url":"https://github.com/yyvhang/IAGNet/blob/HEAD/model/IAGNet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8901b6b5925b9d1e"}},{"code_sha256_prefix":"22406a410f1e5e96","entry":"Joint_Region_Alignment","repo":"yyvhang/IAGNet","repo_kind":"official","path":"model/IAGNet.py","file_url":"https://github.com/yyvhang/IAGNet/blob/HEAD/model/IAGNet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"22406a410f1e5e96"}},{"code_sha256_prefix":"bff3ef0d16e7f5e7","entry":"PointNetFeaturePropagation","repo":"yyvhang/IAGNet","repo_kind":"official","path":"model/IAGNet.py","file_url":"https://github.com/yyvhang/IAGNet/blob/HEAD/model/IAGNet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bff3ef0d16e7f5e7"}},{"code_sha256_prefix":"203605bab5a93c82","entry":"query_ball_point","repo":"yyvhang/IAGNet","repo_kind":"official","path":"model/IAGNet.py","file_url":"https://github.com/yyvhang/IAGNet/blob/HEAD/model/IAGNet.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"203605bab5a93c82"}},{"code_sha256_prefix":"4b492b12ea1918b8","entry":"square_distance","repo":"yyvhang/IAGNet","repo_kind":"official","path":"model/IAGNet.py","file_url":"https://github.com/yyvhang/IAGNet/blob/HEAD/model/IAGNet.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4b492b12ea1918b8"}},{"code_sha256_prefix":"e49d8b4b1a36af50","entry":"Affordance_Revealed_Module","repo":"yyvhang/IAGNet","repo_kind":"official","path":"model/IAGNet.py","file_url":"https://github.com/yyvhang/IAGNet/blob/HEAD/model/IAGNet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e49d8b4b1a36af50"}},{"code_sha256_prefix":"bf0f05da5351fe31","entry":"Cross_Attention","repo":"yyvhang/IAGNet","repo_kind":"official","path":"model/IAGNet.py","file_url":"https://github.com/yyvhang/IAGNet/blob/HEAD/model/IAGNet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bf0f05da5351fe31"}},{"code_sha256_prefix":"1f437c8fb2d24235","entry":"Decoder","repo":"yyvhang/IAGNet","repo_kind":"official","path":"model/IAGNet.py","file_url":"https://github.com/yyvhang/IAGNet/blob/HEAD/model/IAGNet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1f437c8fb2d24235"}},{"code_sha256_prefix":"0abe82543b9b4cef","entry":"IAG","repo":"yyvhang/IAGNet","repo_kind":"official","path":"model/IAGNet.py","file_url":"https://github.com/yyvhang/IAGNet/blob/HEAD/model/IAGNet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0abe82543b9b4cef"}},{"code_sha256_prefix":"b727c1f154254c0a","entry":"PointNetSetAbstractionMsg","repo":"yyvhang/IAGNet","repo_kind":"official","path":"model/IAGNet.py","file_url":"https://github.com/yyvhang/IAGNet/blob/HEAD/model/IAGNet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b727c1f154254c0a"}},{"code_sha256_prefix":"ace7182e44f0c7f0","entry":"Point_Encoder","repo":"yyvhang/IAGNet","repo_kind":"official","path":"model/IAGNet.py","file_url":"https://github.com/yyvhang/IAGNet/blob/HEAD/model/IAGNet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ace7182e44f0c7f0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}