{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multi-hypotheses-conditioned-point-cloud","title":"Multi-hypotheses Conditioned Point Cloud Diffusion for 3D Human Reconstruction from Occluded Images","arxiv_id":"2409.18364","date":"2024-09-27","proceeding":null,"authors":["Donghwan Kim","Tae-Kyun Kim"],"abstract":"3D human shape reconstruction under severe occlusion due to human-object or human-human interaction is a challenging problem. Parametric models i.e., SMPL(-X), which are based on the statistics across human shapes, can represent whole human body shapes but are limited to minimally-clothed human shapes. Implicit-function-based methods extract features from the parametric models to employ prior knowledge of human bodies and can capture geometric details such as clothing and hair. However, they often struggle to handle misaligned parametric models and inpaint occluded regions given a single RGB image. In this work, we propose a novel pipeline, MHCDIFF, Multi-hypotheses Conditioned Point Cloud Diffusion, composed of point cloud diffusion conditioned on probabilistic distributions for pixel-aligned detailed 3D human reconstruction under occlusion. Compared to previous implicit-function-based methods, the point cloud diffusion model can capture the global consistent features to generate the occluded regions, and the denoising process corrects the misaligned SMPL meshes. The core of MHCDIFF is extracting local features from multiple hypothesized SMPL(-X) meshes and aggregating the set of features to condition the diffusion model. In the experiments on CAPE and MultiHuman datasets, the proposed method outperforms various SOTA methods based on SMPL, implicit functions, point cloud diffusion, and their combined, under synthetic and real occlusions. Our code is publicly available at https://donghwankim0101.github.io/projects/mhcdiff/ .","url_abs":"https://arxiv.org/abs/2409.18364v3","url_pdf":"https://arxiv.org/pdf/2409.18364v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multi-hypotheses-conditioned-point-cloud","repo_url":"https://github.com/DonghwanKIM0101/MHCDIFF","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"3d-human-reconstruction","task_name":"3D Human Reconstruction"},{"task_slug":"denoising","task_name":"Denoising"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2409.18364","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.18364"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/DonghwanKIM0101/MHCDIFF","reach":{"status":"ok"}}],"summary":{"ran_fixture":2,"ran":3},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"449a0265144f6530","entry":"index_points","repo":"DonghwanKIM0101/MHCDIFF","repo_kind":"official","path":"fps_sampling_thuman.py","file_url":"https://github.com/DonghwanKIM0101/MHCDIFF/blob/HEAD/fps_sampling_thuman.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":2,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"449a0265144f6530"}},{"code_sha256_prefix":"0eeb77e85d774c36","entry":"compute_distance_transform","repo":"DonghwanKIM0101/MHCDIFF","repo_kind":"official","path":"utils/model_utils.py","file_url":"https://github.com/DonghwanKIM0101/MHCDIFF/blob/HEAD/utils/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0eeb77e85d774c36"}},{"code_sha256_prefix":"955167c60edf7817","entry":"default","repo":"DonghwanKIM0101/MHCDIFF","repo_kind":"official","path":"utils/model_utils.py","file_url":"https://github.com/DonghwanKIM0101/MHCDIFF/blob/HEAD/utils/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"955167c60edf7817"}},{"code_sha256_prefix":"8016bf9f4fcc1ce4","entry":"farthest_point_sample","repo":"DonghwanKIM0101/MHCDIFF","repo_kind":"official","path":"fps_sampling_thuman.py","file_url":"https://github.com/DonghwanKIM0101/MHCDIFF/blob/HEAD/fps_sampling_thuman.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8016bf9f4fcc1ce4"}},{"code_sha256_prefix":"99866b5b5c8de318","entry":"make_rotate","repo":"DonghwanKIM0101/MHCDIFF","repo_kind":"official","path":"render_multi.py","file_url":"https://github.com/DonghwanKIM0101/MHCDIFF/blob/HEAD/render_multi.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"99866b5b5c8de318"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}