{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unihpe-towards-unified-human-pose-estimation","title":"UniHPE: Towards Unified Human Pose Estimation via Contrastive Learning","arxiv_id":"2311.16477","date":"2023-11-24","proceeding":null,"authors":["Zhongyu Jiang","Wenhao Chai","Lei LI","Zhuoran Zhou","Cheng-Yen Yang","Jenq-Neng Hwang"],"abstract":"In recent times, there has been a growing interest in developing effective perception techniques for combining information from multiple modalities. This involves aligning features obtained from diverse sources to enable more efficient training with larger datasets and constraints, as well as leveraging the wealth of information contained in each modality. 2D and 3D Human Pose Estimation (HPE) are two critical perceptual tasks in computer vision, which have numerous downstream applications, such as Action Recognition, Human-Computer Interaction, Object tracking, etc. Yet, there are limited instances where the correlation between Image and 2D/3D human pose has been clearly researched using a contrastive paradigm. In this paper, we propose UniHPE, a unified Human Pose Estimation pipeline, which aligns features from all three modalities, i.e., 2D human pose estimation, lifting-based and image-based 3D human pose estimation, in the same pipeline. To align more than two modalities at the same time, we propose a novel singular value based contrastive learning loss, which better aligns different modalities and further boosts the performance. In our evaluation, UniHPE achieves remarkable performance metrics: MPJPE $50.5$mm on the Human3.6M dataset and PAMPJPE $51.6$mm on the 3DPW dataset. Our proposed method holds immense potential to advance the field of computer vision and contribute to various applications.","url_abs":"https://arxiv.org/abs/2311.16477v1","url_pdf":"https://arxiv.org/pdf/2311.16477v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"2d-human-pose-estimation","task_name":"2D Human Pose Estimation"},{"task_slug":"3d-human-pose-estimation","task_name":"3D Human Pose Estimation"},{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"object-tracking","task_name":"Object Tracking"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-human-pose-estimation-on-3dpw","task":"3D Human Pose Estimation","dataset":"3DPW","model":"UniHPE (GT)","rank_in_archive_order":76,"of":119,"metrics":{"PA-MPJPE":"51.6"},"uses_additional_data":false},{"leaderboard":"/sota/3d-human-pose-estimation-on-3dpw","task":"3D Human Pose Estimation","dataset":"3DPW","model":"UniHPE-w48","rank_in_archive_order":100,"of":119,"metrics":{"PA-MPJPE":"65.7"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2311.16477","atlas_url":"https://app.syntology.ai/?focus=2311.16477","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}