{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/joint-reconstruction-of-3d-human-and-object","title":"Joint Reconstruction of 3D Human and Object via Contact-Based Refinement Transformer","arxiv_id":"2404.04819","date":"2024-04-07","proceeding":"CVPR 2024 1","authors":["Hyeongjin Nam","Daniel Sungho Jung","Gyeongsik Moon","Kyoung Mu Lee"],"abstract":"Human-object contact serves as a strong cue to understand how humans physically interact with objects. Nevertheless, it is not widely explored to utilize human-object contact information for the joint reconstruction of 3D human and object from a single image. In this work, we present a novel joint 3D human-object reconstruction method (CONTHO) that effectively exploits contact information between humans and objects. There are two core designs in our system: 1) 3D-guided contact estimation and 2) contact-based 3D human and object refinement. First, for accurate human-object contact estimation, CONTHO initially reconstructs 3D humans and objects and utilizes them as explicit 3D guidance for contact estimation. Second, to refine the initial reconstructions of 3D human and object, we propose a novel contact-based refinement Transformer that effectively aggregates human features and object features based on the estimated human-object contact. The proposed contact-based refinement prevents the learning of erroneous correlation between human and object, which enables accurate 3D reconstruction. As a result, our CONTHO achieves state-of-the-art performance in both human-object contact estimation and joint reconstruction of 3D human and object. The code is publicly available at https://github.com/dqj5182/CONTHO_RELEASE.","url_abs":"https://arxiv.org/abs/2404.04819v1","url_pdf":"https://arxiv.org/pdf/2404.04819v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"joint-reconstruction-of-3d-human-and-object","repo_url":"https://github.com/dqj5182/contho_release","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"3d-human-reconstruction","task_name":"3D Human Reconstruction"},{"task_slug":"3d-object-reconstruction","task_name":"3D Object Reconstruction"},{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"},{"task_slug":"contact-detection","task_name":"Contact Detection"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-reconstruction","task_name":"Object Reconstruction"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":null,"method_name":"Contact Information"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-human-reconstruction-on-behave","task":"3D Human Reconstruction","dataset":"BEHAVE","model":"CONTHO","rank_in_archive_order":1,"of":3,"metrics":{"Chamfer Distance":"4.99"},"uses_additional_data":false},{"leaderboard":"/sota/3d-object-reconstruction-on-behave","task":"3D Object Reconstruction","dataset":"BEHAVE","model":"CONTHO","rank_in_archive_order":1,"of":3,"metrics":{"Chamfer Distance":"8.42"},"uses_additional_data":false},{"leaderboard":"/sota/contact-detection-on-behave","task":"Contact Detection","dataset":"BEHAVE","model":"CONTHO","rank_in_archive_order":1,"of":4,"metrics":{"Precision":"0.754","Recall":"0.587"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.04819","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.04819"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dqj5182/contho_release","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_draft_wrong":1,"ran":3,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"dqj5182/contho_release","repo_kind":"official","path":"lib/models/resnet.py","file_url":"https://github.com/dqj5182/contho_release/blob/HEAD/lib/models/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"622140af456ff5d2","entry":"make_conv_layers","repo":"dqj5182/contho_release","repo_kind":"official","path":"lib/models/layer.py","file_url":"https://github.com/dqj5182/contho_release/blob/HEAD/lib/models/layer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"622140af456ff5d2"}},{"code_sha256_prefix":"3a26e5305cd9fa2b","entry":"make_deconv_layers","repo":"dqj5182/contho_release","repo_kind":"official","path":"lib/models/layer.py","file_url":"https://github.com/dqj5182/contho_release/blob/HEAD/lib/models/layer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3a26e5305cd9fa2b"}},{"code_sha256_prefix":"6309528f3f76d031","entry":"make_linear_layers","repo":"dqj5182/contho_release","repo_kind":"official","path":"lib/models/layer.py","file_url":"https://github.com/dqj5182/contho_release/blob/HEAD/lib/models/layer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"6309528f3f76d031"}},{"code_sha256_prefix":"3197dd11cf766894","entry":"create_prior","repo":"dqj5182/contho_release","repo_kind":"official","path":"lib/core/prior.py","file_url":"https://github.com/dqj5182/contho_release/blob/HEAD/lib/core/prior.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3197dd11cf766894"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}