Papers › Towards unconstrained joint hand-object reconstruction from RGB videos
Towards unconstrained joint hand-object reconstruction from RGB videos
Yana Hasson, Gül Varol, Ivan Laptev, Cordelia Schmid
Our work aims to obtain 3D reconstruction of hands and manipulated objects from monocular videos. Reconstructing hand-object manipulations holds a great potential for robotics and learning from human demonstrations. The supervised learning approach to this problem, however, requires 3D supervision and remains limited to constrained laboratory settings and simulators for which 3D ground truth is available. In this paper we first propose a learning-free fitting approach for hand-object reconstruction which can seamlessly handle two-hand object interactions. Our method relies on cues obtained with common methods for object detection, hand pose estimation and instance segmentation. We quantitatively evaluate our approach and show that it can be applied to datasets with varying levels of difficulty for which training data is unavailable.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| 3D Hand Pose Estimation | HO-3D v2 | HOR | PA-MPJPE (mm) | 12.0 | #22 of 24 | Archive leaderboard | report |
| hand-object pose | DexYCB | UHO | ADD-S | - | #8 of 9 | Archive leaderboard | report |
| hand-object pose | DexYCB | UHO | Average MPJPE (mm) | 18.8 | #8 of 9 | Archive leaderboard | report |
| hand-object pose | DexYCB | UHO | MCE | 52.5 | #8 of 9 | Archive leaderboard | report |
| hand-object pose | DexYCB | UHO | OCE | - | #8 of 9 | Archive leaderboard | report |
| hand-object pose | DexYCB | UHO | Procrustes-Aligned MPJPE | - | #8 of 9 | Archive leaderboard | report |
| hand-object pose | HO-3D v2 | HOR | ADD-S | 40.0 | #5 of 9 | Archive leaderboard | report |
| hand-object pose | HO-3D v2 | HOR | Average MPJPE (mm) | - | #5 of 9 | Archive leaderboard | report |
| hand-object pose | HO-3D v2 | HOR | OME | 80.0 | #5 of 9 | Archive leaderboard | report |
| hand-object pose | HO-3D v2 | HOR | PA-MPJPE | 12.0 | #5 of 9 | Archive leaderboard | report |
| hand-object pose | HO-3D v2 | HOR | ST-MPJPE | 26.8 | #5 of 9 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections