{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mug-multi-human-graph-network-for-3d-mesh","title":"MUG: Multi-human Graph Network for 3D Mesh Reconstruction from 2D Pose","arxiv_id":"2205.12583","date":"2022-05-25","proceeding":null,"authors":["Chenyan Wu","Yandong Li","Xianfeng Tang","James Wang"],"abstract":"Reconstructing multi-human body mesh from a single monocular image is an important but challenging computer vision problem. In addition to the individual body mesh models, we need to estimate relative 3D positions among subjects to generate a coherent representation. In this work, through a single graph neural network, named MUG (Multi-hUman Graph network), we construct coherent multi-human meshes using only multi-human 2D pose as input. Compared with existing methods, which adopt a detection-style pipeline (i.e., extracting image features and then locating human instances and recovering body meshes from that) and suffer from the significant domain gap between lab-collected training datasets and in-the-wild testing datasets, our method benefits from the 2D pose which has a relatively consistent geometric property across datasets. Our method works like the following: First, to model the multi-human environment, it processes multi-human 2D poses and builds a novel heterogeneous graph, where nodes from different people and within one person are connected to capture inter-human interactions and draw the body geometry (i.e., skeleton and mesh structure). Second, it employs a dual-branch graph neural network structure -- one for predicting inter-human depth relation and the other one for predicting root-joint-relative mesh coordinates. Finally, the entire multi-human 3D meshes are constructed by combining the output from both branches. Extensive experiments demonstrate that MUG outperforms previous multi-human mesh estimation methods on standard 3D human benchmarks -- Panoptic, MuPoTS-3D and 3DPW.","url_abs":"https://arxiv.org/abs/2205.12583v3","url_pdf":"https://arxiv.org/pdf/2205.12583v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"3d-human-pose-estimation","task_name":"3D Human Pose Estimation"},{"task_slug":"3d-multi-person-human-pose-estimation","task_name":"3D Multi-Person Human Pose Estimation"},{"task_slug":"3d-multi-person-pose-estimation","task_name":"3D Multi-Person Pose Estimation"},{"task_slug":"graph-neural-network","task_name":"Graph Neural Network"}],"methods":[{"method_slug":"graph-neural-network","method_name":"Graph Neural Network"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-human-pose-estimation-on-3dpw","task":"3D Human Pose Estimation","dataset":"3DPW","model":"MUG","rank_in_archive_order":57,"of":119,"metrics":{"MPJPE":"87","MPVPE":"106.2","PA-MPJPE":"60.5"},"uses_additional_data":false},{"leaderboard":"/sota/3d-human-pose-estimation-on-cmu-panoptic","task":"3D Human Pose Estimation","dataset":"Panoptic","model":"MUG","rank_in_archive_order":9,"of":9,"metrics":{"Average MPJPE (mm)":"127.8"},"uses_additional_data":false},{"leaderboard":"/sota/3d-multi-person-human-pose-estimation-on","task":"3D Multi-Person Pose Estimation","dataset":"MuPoTS-3D","model":"MUG","rank_in_archive_order":5,"of":10,"metrics":{"3DPCK":"76.27"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}