{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-to-regress-bodies-from-images-using-1","title":"Learning to Regress Bodies from Images using Differentiable Semantic Rendering","arxiv_id":"2110.03480","date":"2021-10-07","proceeding":"ICCV 2021 10","authors":["Sai Kumar Dwivedi","Nikos Athanasiou","Muhammed Kocabas","Michael J. Black"],"abstract":"Learning to regress 3D human body shape and pose (e.g.~SMPL parameters) from monocular images typically exploits losses on 2D keypoints, silhouettes, and/or part-segmentation when 3D training data is not available. Such losses, however, are limited because 2D keypoints do not supervise body shape and segmentations of people in clothing do not match projected minimally-clothed SMPL shapes. To exploit richer image information about clothed people, we introduce higher-level semantic information about clothing to penalize clothed and non-clothed regions of the image differently. To do so, we train a body regressor using a novel Differentiable Semantic Rendering - DSR loss. For Minimally-Clothed regions, we define the DSR-MC loss, which encourages a tight match between a rendered SMPL body and the minimally-clothed regions of the image. For clothed regions, we define the DSR-C loss to encourage the rendered SMPL body to be inside the clothing mask. To ensure end-to-end differentiable training, we learn a semantic clothing prior for SMPL vertices from thousands of clothed human scans. We perform extensive qualitative and quantitative experiments to evaluate the role of clothing semantics on the accuracy of 3D human pose and shape estimation. We outperform all previous state-of-the-art methods on 3DPW and Human3.6M and obtain on par results on MPI-INF-3DHP. Code and trained models are available for research at https://dsr.is.tue.mpg.de/.","url_abs":"https://arxiv.org/abs/2110.03480v2","url_pdf":"https://arxiv.org/pdf/2110.03480v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-to-regress-bodies-from-images-using-1","repo_url":"https://github.com/saidwivedi/DSR","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3d-human-pose-estimation","task_name":"3D Human Pose Estimation"},{"task_slug":"3d-human-pose-and-shape-estimation","task_name":"3D human pose and shape estimation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-human-pose-estimation-on-mpi-inf-3dhp","task":"3D Human Pose Estimation","dataset":"MPI-INF-3DHP","model":"DSR","rank_in_archive_order":72,"of":108,"metrics":{"MPJPE":"104.7","PA-MPJPE":"66.7"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2110.03480","atlas_url":"https://app.syntology.ai/?focus=2110.03480","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.03480"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/saidwivedi/DSR","reach":null}],"summary":{"ran_draft_wrong":2,"ran_fixture":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"30c4823533e14eef","entry":"distance_transform_loss","repo":"saidwivedi/DSR","repo_kind":"official","path":"dsr/semantic_rendering/loss.py","file_url":"https://github.com/saidwivedi/DSR/blob/HEAD/dsr/semantic_rendering/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"30c4823533e14eef"}},{"code_sha256_prefix":"a2036b0894b56e19","entry":"dsr_mc_loss","repo":"saidwivedi/DSR","repo_kind":"official","path":"dsr/semantic_rendering/loss.py","file_url":"https://github.com/saidwivedi/DSR/blob/HEAD/dsr/semantic_rendering/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a2036b0894b56e19"}},{"code_sha256_prefix":"77749d9c8224a0a3","entry":"neg_iou_loss","repo":"saidwivedi/DSR","repo_kind":"official","path":"dsr/semantic_rendering/loss.py","file_url":"https://github.com/saidwivedi/DSR/blob/HEAD/dsr/semantic_rendering/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"77749d9c8224a0a3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}