Papers › HOnnotate: A method for 3D Annotation of Hand and Object Poses

HOnnotate: A method for 3D Annotation of Hand and Object Poses

2 Jul 2019CVPR 2020 6arXiv:1907.01481archive 2025-07-28

Shreyas Hampali, Mahdi Rad, Markus Oberweger, Vincent Lepetit

We propose a method for annotating images of a hand manipulating an object with the 3D poses of both the hand and the object, together with a dataset created using this method. Our motivation is the current lack of annotated real images for this problem, as estimating the 3D poses is challenging, mostly because of the mutual occlusions between the hand and the object. To tackle this challenge, we capture sequences with one or several RGB-D cameras and jointly optimize the 3D hand and object poses over all the frames simultaneously. This method allows us to automatically annotate each frame with accurate estimates of the poses, despite large mutual occlusions. With this method, we created HO-3D, the first markerless dataset of color images with 3D annotations for both the hand and object. This dataset is currently made of 77,558 frames, 68 sequences, 10 persons, and 10 objects. Using our dataset, we develop a single RGB image-based method to predict the hand pose when interacting with objects under severe occlusions and show it generalizes to objects not seen in the dataset.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

anilarmagan/HANDS19-Challenge-Toolbox mentioned on GitHubpytorchSyntology: not harvested report
rongakowang/densemutualattention mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Interacting Hand-Object Pose Estimation via Dense Mutual Attention (arXiv:2211.08805; that paper's own run record: 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
shreyashampali/HOnnotate mentioned on GitHubtfSyntology: not harvested report
shreyashampali/ho3d mentioned on GitHubSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

3D Hand Pose EstimationHand Pose EstimationObjectPose EstimationPose Predictionhand-object pose

Datasets

Introduced by this paper, per the archive.

HO-3D v2

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
3D Hand Pose Estimation HO-3D v2 Hampali et al. AUC_J 0.788 #17 of 24 Archive leaderboard report
3D Hand Pose Estimation HO-3D v2 Hampali et al. AUC_V 0.790 #17 of 24 Archive leaderboard report
3D Hand Pose Estimation HO-3D v2 Hampali et al. F@15mm 0.942 #17 of 24 Archive leaderboard report
3D Hand Pose Estimation HO-3D v2 Hampali et al. F@5mm 0.506 #17 of 24 Archive leaderboard report
3D Hand Pose Estimation HO-3D v2 Hampali et al. PA-MPJPE (mm) 10.7 #17 of 24 Archive leaderboard report
3D Hand Pose Estimation HO-3D v2 Hampali et al. PA-MPVPE 10.6 #17 of 24 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections