{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/one-stage-3d-whole-body-mesh-recovery-with","title":"One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer","arxiv_id":"2303.16160","date":"2023-03-28","proceeding":"CVPR 2023 1","authors":["Jing Lin","Ailing Zeng","Haoqian Wang","Lei Zhang","Yu Li"],"abstract":"Whole-body mesh recovery aims to estimate the 3D human body, face, and hands parameters from a single image. It is challenging to perform this task with a single network due to resolution issues, i.e., the face and hands are usually located in extremely small regions. Existing works usually detect hands and faces, enlarge their resolution to feed in a specific network to predict the parameter, and finally fuse the results. While this copy-paste pipeline can capture the fine-grained details of the face and hands, the connections between different parts cannot be easily recovered in late fusion, leading to implausible 3D rotation and unnatural pose. In this work, we propose a one-stage pipeline for expressive whole-body mesh recovery, named OSX, without separate networks for each part. Specifically, we design a Component Aware Transformer (CAT) composed of a global body encoder and a local face/hand decoder. The encoder predicts the body parameters and provides a high-quality feature map for the decoder, which performs a feature-level upsample-crop scheme to extract high-resolution part-specific features and adopt keypoint-guided deformable attention to estimate hand and face precisely. The whole pipeline is simple yet effective without any manual post-processing and naturally avoids implausible prediction. Comprehensive experiments demonstrate the effectiveness of OSX. Lastly, we build a large-scale Upper-Body dataset (UBody) with high-quality 2D and 3D whole-body annotations. It contains persons with partially visible bodies in diverse real-life scenarios to bridge the gap between the basic task and downstream applications.","url_abs":"https://arxiv.org/abs/2303.16160v1","url_pdf":"https://arxiv.org/pdf/2303.16160v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"one-stage-3d-whole-body-mesh-recovery-with","repo_url":"https://github.com/IDEA-Research/OSX","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"3d-human-pose-estimation","task_name":"3D Human Pose Estimation"},{"task_slug":"3d-human-reconstruction","task_name":"3D Human Reconstruction"},{"task_slug":"3d-multi-person-mesh-recovery","task_name":"3D Multi-Person Mesh Recovery"},{"task_slug":"decoder","task_name":"Decoder"}],"methods":[{"method_slug":"aware","method_name":"AWARE"},{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"copy-paste","method_name":"Copy-Paste"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[{"slug":"ubody","name":"UBody","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-human-pose-estimation-on-3dpw","task":"3D Human Pose Estimation","dataset":"3DPW","model":"OSX","rank_in_archive_order":68,"of":119,"metrics":{"MPJPE":"74.7","PA-MPJPE":"45.1"},"uses_additional_data":false},{"leaderboard":"/sota/3d-human-pose-estimation-on-ubody","task":"3D Human Pose Estimation","dataset":"UBody","model":"OSX","rank_in_archive_order":3,"of":4,"metrics":{"PA-PVE-All":"42.2","PA-PVE-Face":"2.0","PA-PVE-Hands":"8.6","PVE-All":"81.9","PVE-Face":"21.2","PVE-Hands":"41.5"},"uses_additional_data":false},{"leaderboard":"/sota/3d-human-reconstruction-on-ehf","task":"3D Human Reconstruction","dataset":"EHF","model":"OSX","rank_in_archive_order":3,"of":3,"metrics":{"MPVPE":"70.8","PA V2V (mm), face":"6","PA V2V (mm), whole body":"48.7"},"uses_additional_data":false},{"leaderboard":"/sota/3d-multi-person-mesh-recovery-on-agora","task":"3D Multi-Person Mesh Recovery","dataset":"AGORA","model":"OSX","rank_in_archive_order":4,"of":7,"metrics":{"B-NMVE":"85.3","F-MVE":"36.2","FB-MVE":"122.8","FB-NMVE":"130.6","LH/RH-MVE":"45.7"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2303.16160","atlas_url":"https://app.syntology.ai/?focus=2303.16160","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.16160"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IDEA-Research/OSX","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"622140af456ff5d2","entry":"make_conv_layers","repo":"IDEA-Research/OSX","repo_kind":"official","path":"common/nets/layer.py","file_url":"https://github.com/IDEA-Research/OSX/blob/HEAD/common/nets/layer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"622140af456ff5d2"}},{"code_sha256_prefix":"3a26e5305cd9fa2b","entry":"make_deconv_layers","repo":"IDEA-Research/OSX","repo_kind":"official","path":"common/nets/layer.py","file_url":"https://github.com/IDEA-Research/OSX/blob/HEAD/common/nets/layer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3a26e5305cd9fa2b"}},{"code_sha256_prefix":"6309528f3f76d031","entry":"make_linear_layers","repo":"IDEA-Research/OSX","repo_kind":"official","path":"common/nets/layer.py","file_url":"https://github.com/IDEA-Research/OSX/blob/HEAD/common/nets/layer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6309528f3f76d031"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}