{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deco-dense-estimation-of-3d-human-scene-1","title":"DECO: Dense Estimation of 3D Human-Scene Contact In The Wild","arxiv_id":"2309.15273","date":"2023-09-26","proceeding":"ICCV 2023 1","authors":["Shashank Tripathi","Agniv Chatterjee","Jean-Claude Passy","Hongwei Yi","Dimitrios Tzionas","Michael J. Black"],"abstract":"Understanding how humans use physical contact to interact with the world is key to enabling human-centric artificial intelligence. While inferring 3D contact is crucial for modeling realistic and physically-plausible human-object interactions, existing methods either focus on 2D, consider body joints rather than the surface, use coarse 3D body regions, or do not generalize to in-the-wild images. In contrast, we focus on inferring dense, 3D contact between the full body surface and objects in arbitrary images. To achieve this, we first collect DAMON, a new dataset containing dense vertex-level contact annotations paired with RGB images containing complex human-object and human-scene contact. Second, we train DECO, a novel 3D contact detector that uses both body-part-driven and scene-context-driven attention to estimate vertex-level contact on the SMPL body. DECO builds on the insight that human observers recognize contact by reasoning about the contacting body parts, their proximity to scene objects, and the surrounding scene context. We perform extensive evaluations of our detector on DAMON as well as on the RICH and BEHAVE datasets. We significantly outperform existing SOTA methods across all benchmarks. We also show qualitatively that DECO generalizes well to diverse and challenging real-world human interactions in natural images. The code, data, and models are available at https://deco.is.tue.mpg.de.","url_abs":"https://arxiv.org/abs/2309.15273v1","url_pdf":"https://arxiv.org/pdf/2309.15273v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deco-dense-estimation-of-3d-human-scene-1","repo_url":"https://github.com/sha2nkt/deco","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"deco-dense-estimation-of-3d-human-scene-1","repo_url":"https://github.com/CarstenEpic/humos","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"contact-detection","task_name":"Contact Detection"},{"task_slug":"dense-contact-estimation","task_name":"Dense contact estimation"},{"task_slug":"human-object-interaction-detection","task_name":"Human-Object Interaction Detection"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[{"slug":"damon","name":"DAMON","full_name":"Dense Annotation of 3D Human Object contact in Natural Images"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/contact-detection-on-behave","task":"Contact Detection","dataset":"BEHAVE","model":"DECO","rank_in_archive_order":2,"of":4,"metrics":{"Precision":"0.638","Recall":"0.337"},"uses_additional_data":false},{"leaderboard":"/sota/dense-contact-estimation-on-mow","task":"Dense contact estimation","dataset":"MOW","model":"DECO","rank_in_archive_order":2,"of":4,"metrics":{"F1-Score":"0.197","Precision":"0.246","Recall":"0.235"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2309.15273","atlas_url":"https://app.syntology.ai/?focus=2309.15273","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.15273"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sha2nkt/deco","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/CarstenEpic/humos","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":6,"ran_draft_wrong":1,"unverified":5},"by_repo_kind":{"official":{"samples":12,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"aa08b1e9b931f4dd","entry":"BasicBlock","repo":"sha2nkt/deco","repo_kind":"official","path":"models/deco.py","file_url":"https://github.com/sha2nkt/deco/blob/HEAD/models/deco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"aa08b1e9b931f4dd"}},{"code_sha256_prefix":"4c099b4d3a311a6f","entry":"Classifier","repo":"sha2nkt/deco","repo_kind":"official","path":"models/deco.py","file_url":"https://github.com/sha2nkt/deco/blob/HEAD/models/deco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4c099b4d3a311a6f"}},{"code_sha256_prefix":"3fc2ad51ca7a1405","entry":"Cross_Att","repo":"sha2nkt/deco","repo_kind":"official","path":"models/deco.py","file_url":"https://github.com/sha2nkt/deco/blob/HEAD/models/deco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3fc2ad51ca7a1405"}},{"code_sha256_prefix":"5f277424861d148f","entry":"Decoder","repo":"sha2nkt/deco","repo_kind":"official","path":"models/deco.py","file_url":"https://github.com/sha2nkt/deco/blob/HEAD/models/deco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5f277424861d148f"}},{"code_sha256_prefix":"a06fe498ccd35dab","entry":"HighResolutionModule","repo":"sha2nkt/deco","repo_kind":"official","path":"models/deco.py","file_url":"https://github.com/sha2nkt/deco/blob/HEAD/models/deco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a06fe498ccd35dab"}},{"code_sha256_prefix":"80c771b3b438e460","entry":"Self_Attn","repo":"sha2nkt/deco","repo_kind":"official","path":"models/deco.py","file_url":"https://github.com/sha2nkt/deco/blob/HEAD/models/deco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"80c771b3b438e460"}},{"code_sha256_prefix":"779c0baa2f2d2439","entry":"get_cfg_defaults","repo":"sha2nkt/deco","repo_kind":"official","path":"models/deco.py","file_url":"https://github.com/sha2nkt/deco/blob/HEAD/models/deco.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"779c0baa2f2d2439"}},{"code_sha256_prefix":"df75e7818b4fed38","entry":"DECO","repo":"sha2nkt/deco","repo_kind":"official","path":"models/deco.py","file_url":"https://github.com/sha2nkt/deco/blob/HEAD/models/deco.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"df75e7818b4fed38"}},{"code_sha256_prefix":"0d8376e8bee064b3","entry":"Encoder","repo":"sha2nkt/deco","repo_kind":"official","path":"models/deco.py","file_url":"https://github.com/sha2nkt/deco/blob/HEAD/models/deco.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"0d8376e8bee064b3"}},{"code_sha256_prefix":"6fb96a9d4d5b6696","entry":"PoseHighResolutionNet","repo":"sha2nkt/deco","repo_kind":"official","path":"models/deco.py","file_url":"https://github.com/sha2nkt/deco/blob/HEAD/models/deco.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"6fb96a9d4d5b6696"}},{"code_sha256_prefix":"8c72b232bd94f8d4","entry":"get_pose_net","repo":"sha2nkt/deco","repo_kind":"official","path":"models/deco.py","file_url":"https://github.com/sha2nkt/deco/blob/HEAD/models/deco.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"8c72b232bd94f8d4"}},{"code_sha256_prefix":"5e2070c53b6e1fcb","entry":"hrnet_w32","repo":"sha2nkt/deco","repo_kind":"official","path":"models/deco.py","file_url":"https://github.com/sha2nkt/deco/blob/HEAD/models/deco.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5e2070c53b6e1fcb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}