{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/end-to-end-recovery-of-human-shape-and-pose","title":"End-to-end Recovery of Human Shape and Pose","arxiv_id":"1712.06584","date":"2017-12-18","proceeding":"CVPR 2018 6","authors":["Angjoo Kanazawa","Michael J. Black","David W. Jacobs","Jitendra Malik"],"abstract":"We describe Human Mesh Recovery (HMR), an end-to-end framework for\nreconstructing a full 3D mesh of a human body from a single RGB image. In\ncontrast to most current methods that compute 2D or 3D joint locations, we\nproduce a richer and more useful mesh representation that is parameterized by\nshape and 3D joint angles. The main objective is to minimize the reprojection\nloss of keypoints, which allow our model to be trained using images in-the-wild\nthat only have ground truth 2D annotations. However, the reprojection loss\nalone leaves the model highly under constrained. In this work we address this\nproblem by introducing an adversary trained to tell whether a human body\nparameter is real or not using a large database of 3D human meshes. We show\nthat HMR can be trained with and without using any paired 2D-to-3D supervision.\nWe do not rely on intermediate 2D keypoint detections and infer 3D pose and\nshape parameters directly from image pixels. Our model runs in real-time given\na bounding box containing the person. We demonstrate our approach on various\nimages in-the-wild and out-perform previous optimization based methods that\noutput 3D meshes and show competitive results on tasks such as 3D joint\nlocation estimation and part segmentation.","url_abs":"http://arxiv.org/abs/1712.06584v2","url_pdf":"http://arxiv.org/pdf/1712.06584v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"end-to-end-recovery-of-human-shape-and-pose","repo_url":"https://github.com/2023-MindSpore-1/ms-code-27","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"mindspore","reach":{"status":"unanswered"}},{"paper_slug":"end-to-end-recovery-of-human-shape-and-pose","repo_url":"https://github.com/Liuxiang0358/HMR","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"mindspore","reach":{"status":"unanswered"}},{"paper_slug":"end-to-end-recovery-of-human-shape-and-pose","repo_url":"https://github.com/MandyMo/pytorch_HMR","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"end-to-end-recovery-of-human-shape-and-pose","repo_url":"https://github.com/ManifoldFR/recvis-project","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"end-to-end-recovery-of-human-shape-and-pose","repo_url":"https://github.com/anilarmagan/HANDS19-Challenge-Toolbox","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"end-to-end-recovery-of-human-shape-and-pose","repo_url":"https://github.com/russoale/hmr2.0","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"end-to-end-recovery-of-human-shape-and-pose","repo_url":"https://github.com/2023-MindSpore-4/Code-5/tree/main/HMR","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"end-to-end-recovery-of-human-shape-and-pose","repo_url":"https://github.com/MindSpore-paper-code-3/code4/tree/main/HMR","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"end-to-end-recovery-of-human-shape-and-pose","repo_url":"https://github.com/akanazawa/hmr","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"end-to-end-recovery-of-human-shape-and-pose","repo_url":"https://github.com/open-mmlab/mmpose","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"3d-hand-pose-estimation","task_name":"3D Hand Pose Estimation"},{"task_slug":"3d-human-pose-estimation","task_name":"3D Human Pose Estimation"},{"task_slug":"3d-human-shape-estimation","task_name":"3D Human Shape Estimation"},{"task_slug":"3d-multi-person-pose-estimation","task_name":"3D Multi-Person Pose Estimation"},{"task_slug":"human-mesh-recovery","task_name":"Human Mesh Recovery"},{"task_slug":"monocular-3d-human-pose-estimation","task_name":"Monocular 3D Human Pose Estimation"},{"task_slug":"multi-hypotheses-3d-human-pose-estimation","task_name":"Multi-Hypotheses 3D Human Pose Estimation"},{"task_slug":"weakly-supervised-3d-human-pose-estimation","task_name":"Weakly-supervised 3D Human Pose Estimation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-human-pose-estimation-on-3dpw","task":"3D Human Pose Estimation","dataset":"3DPW","model":"HMR","rank_in_archive_order":119,"of":119,"metrics":{"Acceleration Error":"37.4","MPJPE":"130.0"},"uses_additional_data":false},{"leaderboard":"/sota/3d-human-pose-estimation-on-agora","task":"3D Human Pose Estimation","dataset":"AGORA","model":"HMR","rank_in_archive_order":11,"of":11,"metrics":{"B-MPJPE":"180.5","B-MVE":"173.6","B-NMJE":"226.0","B-NMVE":"217.0"},"uses_additional_data":false},{"leaderboard":"/sota/3d-human-pose-estimation-on-mpi-inf-3dhp","task":"3D Human Pose Estimation","dataset":"MPI-INF-3DHP","model":"HMR","rank_in_archive_order":81,"of":108,"metrics":{"AUC":"36.5","MPJPE":"124.2","PA-MPJPE":"89.8","PCK":"72.9"},"uses_additional_data":false},{"leaderboard":"/sota/3d-human-shape-estimation-on-ssp-3d","task":"3D Human Shape Estimation","dataset":"SSP-3D","model":"HMR(unpaired)","rank_in_archive_order":7,"of":11,"metrics":{"PVE-T-SC":"20.8","mIOU":"61.0"},"uses_additional_data":false},{"leaderboard":"/sota/3d-human-shape-estimation-on-ssp-3d","task":"3D Human Shape Estimation","dataset":"SSP-3D","model":"HMR","rank_in_archive_order":9,"of":11,"metrics":{"PVE-T-SC":"22.9","mIOU":"69.0"},"uses_additional_data":false},{"leaderboard":"/sota/3d-multi-person-pose-estimation-on-agora","task":"3D Multi-Person Pose Estimation","dataset":"AGORA","model":"HMR","rank_in_archive_order":4,"of":4,"metrics":{"B-MPJPE":"180.5","B-MVE":"173.6","B-NMJE":"226.0","B-NMVE":"217.0"},"uses_additional_data":false},{"leaderboard":"/sota/monocular-3d-human-pose-estimation-on-human3","task":"Monocular 3D Human Pose Estimation","dataset":"Human3.6M","model":"HMR","rank_in_archive_order":41,"of":52,"metrics":{"Frames Needed":"1","Need Ground Truth 2D Pose":"No","Use Video Sequence":"No"},"uses_additional_data":false},{"leaderboard":"/sota/multi-hypotheses-3d-human-pose-estimation-on-2","task":"Multi-Hypotheses 3D Human Pose Estimation","dataset":"AH36M","model":"HMR (2D Vis, by MHEntropy)","rank_in_archive_order":8,"of":10,"metrics":{"Best-Hypothesis MPJPE (n = 25)":"-","Best-Hypothesis PMPJPE (n = 25)":"85.2","H36M PMPJPE (n = 1)":"67.4","H36M PMPJPE (n = 25)":"67.4","Most-Likely Hypothesis PMPJPE (n = 1)":"85.2"},"uses_additional_data":false},{"leaderboard":"/sota/multi-hypotheses-3d-human-pose-estimation-on-2","task":"Multi-Hypotheses 3D Human Pose Estimation","dataset":"AH36M","model":"HMR","rank_in_archive_order":9,"of":10,"metrics":{"Best-Hypothesis MPJPE (n = 25)":"-","Best-Hypothesis PMPJPE (n = 25)":"-","H36M PMPJPE (n = 1)":"56.8","H36M PMPJPE (n = 25)":"56.8","Most-Likely Hypothesis PMPJPE (n = 1)":"-"},"uses_additional_data":false},{"leaderboard":"/sota/weakly-supervised-3d-human-pose-estimation-on","task":"Weakly-supervised 3D Human Pose Estimation","dataset":"Human3.6M","model":"Kanzawa et al.","rank_in_archive_order":33,"of":33,"metrics":{"3D Annotations":"No"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1712.06584","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}