{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/handbooster-boosting-3d-hand-mesh","title":"HandBooster: Boosting 3D Hand-Mesh Reconstruction by Conditional Synthesis and Sampling of Hand-Object Interactions","arxiv_id":"2403.18575","date":"2024-03-27","proceeding":"CVPR 2024 1","authors":["Hao Xu","Haipeng Li","Yinqiao Wang","Shuaicheng Liu","Chi-Wing Fu"],"abstract":"Reconstructing 3D hand mesh robustly from a single image is very challenging, due to the lack of diversity in existing real-world datasets. While data synthesis helps relieve the issue, the syn-to-real gap still hinders its usage. In this work, we present HandBooster, a new approach to uplift the data diversity and boost the 3D hand-mesh reconstruction performance by training a conditional generative space on hand-object interactions and purposely sampling the space to synthesize effective data samples. First, we construct versatile content-aware conditions to guide a diffusion model to produce realistic images with diverse hand appearances, poses, views, and backgrounds; favorably, accurate 3D annotations are obtained for free. Then, we design a novel condition creator based on our similarity-aware distribution sampling strategies to deliberately find novel and realistic interaction poses that are distinctive from the training set. Equipped with our method, several baselines can be significantly improved beyond the SOTA on the HO3D and DexYCB benchmarks. Our code will be released on https://github.com/hxwork/HandBooster_Pytorch.","url_abs":"https://arxiv.org/abs/2403.18575v1","url_pdf":"https://arxiv.org/pdf/2403.18575v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"handbooster-boosting-3d-hand-mesh","repo_url":"https://github.com/hxwork/handbooster_pytorch","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"3d-hand-pose-estimation","task_name":"3D Hand Pose Estimation"},{"task_slug":"diversity","task_name":"Diversity"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-hand-pose-estimation-on-ho-3d","task":"3D Hand Pose Estimation","dataset":"HO-3D v2","model":"HandBooster","rank_in_archive_order":5,"of":24,"metrics":{"AUC_J":"0.836","AUC_V":"0.832","F@15mm":"0.972","F@5mm":"0.585","PA-MPJPE (mm)":"8.2","PA-MPVPE":"8.4"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2403.18575","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.18575"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hxwork/handbooster_pytorch","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/hxwork/HandBooster_Pytorch","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":1,"ran":7,"ran_violates":2},"by_repo_kind":{"official":{"samples":10,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7f1040f5e3991d5e","entry":"identity","repo":"hxwork/HandBooster_Pytorch","repo_kind":"official","path":"HandObjectImageSynthesizer/denoising_diffusion_pytorch/classifier_free_guidance_normal_wloss_grcond.py","file_url":"https://github.com/hxwork/HandBooster_Pytorch/blob/HEAD/HandObjectImageSynthesizer/denoising_diffusion_pytorch/classifier_free_guidance_normal_wloss_grcond.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":2,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f1040f5e3991d5e"}},{"code_sha256_prefix":"38a4447a0132916a","entry":"align_w_scale","repo":"hxwork/HandBooster_Pytorch","repo_kind":"official","path":"HandReconstruction/loss/loss.py","file_url":"https://github.com/hxwork/HandBooster_Pytorch/blob/HEAD/HandReconstruction/loss/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"38a4447a0132916a"}},{"code_sha256_prefix":"ad43d63d6e052f6c","entry":"convert_image_to_fn","repo":"hxwork/HandBooster_Pytorch","repo_kind":"official","path":"HandObjectImageSynthesizer/dataset/DexYCB.py","file_url":"https://github.com/hxwork/HandBooster_Pytorch/blob/HEAD/HandObjectImageSynthesizer/dataset/DexYCB.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad43d63d6e052f6c"}},{"code_sha256_prefix":"d1ef6b8cb9a28a53","entry":"default","repo":"hxwork/HandBooster_Pytorch","repo_kind":"official","path":"HandObjectImageSynthesizer/denoising_diffusion_pytorch/classifier_free_guidance_normal_wloss_grcond.py","file_url":"https://github.com/hxwork/HandBooster_Pytorch/blob/HEAD/HandObjectImageSynthesizer/denoising_diffusion_pytorch/classifier_free_guidance_normal_wloss_grcond.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d1ef6b8cb9a28a53"}},{"code_sha256_prefix":"608e364a9d2376a3","entry":"exists","repo":"hxwork/HandBooster_Pytorch","repo_kind":"official","path":"HandObjectImageSynthesizer/denoising_diffusion_pytorch/classifier_free_guidance_normal_wloss_grcond.py","file_url":"https://github.com/hxwork/HandBooster_Pytorch/blob/HEAD/HandObjectImageSynthesizer/denoising_diffusion_pytorch/classifier_free_guidance_normal_wloss_grcond.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"608e364a9d2376a3"}},{"code_sha256_prefix":"41843763717707cc","entry":"make_conv1d_layers","repo":"hxwork/HandBooster_Pytorch","repo_kind":"official","path":"HandReconstruction/model/layer.py","file_url":"https://github.com/hxwork/HandBooster_Pytorch/blob/HEAD/HandReconstruction/model/layer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"41843763717707cc"}},{"code_sha256_prefix":"622140af456ff5d2","entry":"make_conv_layers","repo":"hxwork/HandBooster_Pytorch","repo_kind":"official","path":"HandReconstruction/model/layer.py","file_url":"https://github.com/hxwork/HandBooster_Pytorch/blob/HEAD/HandReconstruction/model/layer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"622140af456ff5d2"}},{"code_sha256_prefix":"6309528f3f76d031","entry":"make_linear_layers","repo":"hxwork/HandBooster_Pytorch","repo_kind":"official","path":"HandReconstruction/model/layer.py","file_url":"https://github.com/hxwork/HandBooster_Pytorch/blob/HEAD/HandReconstruction/model/layer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6309528f3f76d031"}},{"code_sha256_prefix":"e6c4724cfa90f7d7","entry":"tensor_gpu","repo":"hxwork/HandBooster_Pytorch","repo_kind":"official","path":"HandObjectImageSynthesizer/denoising_diffusion_pytorch/tools.py","file_url":"https://github.com/hxwork/HandBooster_Pytorch/blob/HEAD/HandObjectImageSynthesizer/denoising_diffusion_pytorch/tools.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e6c4724cfa90f7d7"}},{"code_sha256_prefix":"cbcae1ad610bc215","entry":"torch_align_w_scale","repo":"hxwork/HandBooster_Pytorch","repo_kind":"official","path":"HandReconstruction/loss/loss.py","file_url":"https://github.com/hxwork/HandBooster_Pytorch/blob/HEAD/HandReconstruction/loss/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cbcae1ad610bc215"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}