Papers › Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstruction
Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstruction
Xiao Tang, Tianyu Wang, Chi-Wing Fu
3D hand-mesh reconstruction from RGB images facilitates many applications, including augmented reality (AR). However, this requires not only real-time speed and accurate hand pose and shape but also plausible mesh-image alignment. While existing works already achieve promising results, meeting all three requirements is very challenging. This paper presents a novel pipeline by decoupling the hand-mesh reconstruction task into three stages: a joint stage to predict hand joints and segmentation; a mesh stage to predict a rough hand mesh; and a refine stage to fine-tune it with an offset mesh for mesh-image alignment. With careful design in the network structure and in the loss functions, we can promote high-quality finger-level mesh-image alignment and drive the models together to deliver real-time predictions. Extensive quantitative and qualitative results on benchmark datasets demonstrate that the quality of our results outperforms the state-of-the-art methods on hand-mesh/pose precision and hand-image alignment. In the end, we also showcase several real-time AR scenarios.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| 3D Hand Pose Estimation | FreiHAND | Tang et al. | PA-F@15mm | 0.981 | #19 of 33 | Archive leaderboard | report |
| 3D Hand Pose Estimation | FreiHAND | Tang et al. | PA-F@5mm | 0.724 | #19 of 33 | Archive leaderboard | report |
| 3D Hand Pose Estimation | FreiHAND | Tang et al. | PA-MPJPE | 6.7 | #19 of 33 | Archive leaderboard | report |
| 3D Hand Pose Estimation | FreiHAND | Tang et al. | PA-MPVPE | 6.7 | #19 of 33 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections