{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/extpose-robust-and-coherent-pose-estimation","title":"ExtPose: Robust and Coherent Pose Estimation by Extending ViTs","arxiv_id":null,"date":"2025-06-18","proceeding":"International Conference on Machine Learning 2025 6","authors":["Rongyu Chen","Li'an Zhuo","Linlin Yang","Qi Wang","Liefeng Bo","Bang Zhang","Angela Yao"],"abstract":"Vision Transformers (ViT) are remarkable at 3D pose estimation, yet they still encounter certain challenges. One issue is that the popular ViT architecture for pose estimation is limited to images and lacks temporal information. Another challenge is that the prediction often fails to maintain pixel alignment with the original images. To address these issues, we propose a systematic framework for 3D pose estimation, called ExtPose. ExtPose extends image ViT to the challenging scenario and video setting by taking in additional 2D pose evidence and capturing temporal information in a full attention-based manner. We use 2D human skeleton images to integrate structured 2D pose information. By sharing parameters and attending across modalities and frames, we enhance the consistency between 3D poses and 2D videos without introducing additional parameters. We achieve state-of-the-art (SOTA) performance on multiple human and hand pose estimation benchmarks with substantial improvements to 34.0mm (-23%) on 3DPW and 4.9mm (-18%) on FreiHAND in PA-MPJPE over the other ViT-based methods respectively.","url_abs":"https://openreview.net/forum?id=hm9FNEZZ6z","url_pdf":"https://openreview.net/pdf?id=hm9FNEZZ6z","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"3d-hand-pose-estimation","task_name":"3D Hand Pose Estimation"},{"task_slug":"3d-human-pose-estimation","task_name":"3D Human Pose Estimation"},{"task_slug":"3d-pose-estimation","task_name":"3D Pose Estimation"},{"task_slug":"hand-pose-estimation","task_name":"Hand Pose Estimation"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-hand-pose-estimation-on-freihand","task":"3D Hand Pose Estimation","dataset":"FreiHAND","model":"ExtPose","rank_in_archive_order":1,"of":33,"metrics":{"PA-F@15mm":"0.993","PA-F@5mm":"0.823","PA-MPJPE":"4.9","PA-MPVPE":"5.1"},"uses_additional_data":false},{"leaderboard":"/sota/3d-hand-pose-estimation-on-hint-hand","task":"3D Hand Pose Estimation","dataset":"HInt: Hand Interactions in the wild","model":"ExtPose*","rank_in_archive_order":1,"of":10,"metrics":{"PCK@0.05 (New Days) All":"59.6","PCK@0.05 (VISOR) All":"61.1"},"uses_additional_data":false},{"leaderboard":"/sota/3d-hand-pose-estimation-on-ho-3d","task":"3D Hand Pose Estimation","dataset":"HO-3D v2","model":"ExtPose (T=16)","rank_in_archive_order":1,"of":24,"metrics":{"AUC_J":"0.863","AUC_V":"0.856","F@15mm":"0.991","F@5mm":"0.667","PA-MPJPE (mm)":"6.9","PA-MPVPE":"7.3"},"uses_additional_data":false},{"leaderboard":"/sota/3d-human-pose-estimation-on-3dpw","task":"3D Human Pose Estimation","dataset":"3DPW","model":"ExtPose (-a, T=16)","rank_in_archive_order":1,"of":119,"metrics":{"MPJPE":"54.2","MPVPE":"67.5","PA-MPJPE":"34.0"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}