{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/neural-interactive-keypoint-detection","title":"Neural Interactive Keypoint Detection","arxiv_id":"2308.10174","date":"2023-08-20","proceeding":"ICCV 2023 1","authors":["Jie Yang","Ailing Zeng","Feng Li","Shilong Liu","Ruimao Zhang","Lei Zhang"],"abstract":"This work proposes an end-to-end neural interactive keypoint detection framework named Click-Pose, which can significantly reduce more than 10 times labeling costs of 2D keypoint annotation compared with manual-only annotation. Click-Pose explores how user feedback can cooperate with a neural keypoint detector to correct the predicted keypoints in an interactive way for a faster and more effective annotation process. Specifically, we design the pose error modeling strategy that inputs the ground truth pose combined with four typical pose errors into the decoder and trains the model to reconstruct the correct poses, which enhances the self-correction ability of the model. Then, we attach an interactive human-feedback loop that allows receiving users' clicks to correct one or several predicted keypoints and iteratively utilizes the decoder to update all other keypoints with a minimum number of clicks (NoC) for efficient annotation. We validate Click-Pose in in-domain, out-of-domain scenes, and a new task of keypoint adaptation. For annotation, Click-Pose only needs 1.97 and 6.45 NoC@95 (at precision 95%) on COCO and Human-Art, reducing 31.4% and 36.3% efforts than the SOTA model (ViTPose) with manual correction, respectively. Besides, without user clicks, Click-Pose surpasses the previous end-to-end model by 1.4 AP on COCO and 3.0 AP on Human-Art. The code is available at https://github.com/IDEA-Research/Click-Pose.","url_abs":"https://arxiv.org/abs/2308.10174v1","url_pdf":"https://arxiv.org/pdf/2308.10174v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"neural-interactive-keypoint-detection","repo_url":"https://github.com/idea-research/click-pose","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"keypoint-detection","task_name":"Keypoint Detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2308.10174","atlas_url":"https://app.syntology.ai/?focus=2308.10174","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.10174"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/idea-research/click-pose","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":2,"ran_honours":1,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":8,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"26d9f651bd1c2bf2","entry":"check_and_copy","repo":"idea-research/click-pose","repo_kind":"official","path":"datasets/data_util.py","file_url":"https://github.com/idea-research/click-pose/blob/HEAD/datasets/data_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"26d9f651bd1c2bf2"}},{"code_sha256_prefix":"f31a58bf6457ced5","entry":"convert_to_xywh","repo":"idea-research/click-pose","repo_kind":"official","path":"datasets/coco_eval.py","file_url":"https://github.com/idea-research/click-pose/blob/HEAD/datasets/coco_eval.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f31a58bf6457ced5"}},{"code_sha256_prefix":"9d7d26ec4b3dbe75","entry":"copytree","repo":"idea-research/click-pose","repo_kind":"official","path":"datasets/data_util.py","file_url":"https://github.com/idea-research/click-pose/blob/HEAD/datasets/data_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"9d7d26ec4b3dbe75"}},{"code_sha256_prefix":"fe0ddcc2d420c9a0","entry":"evaluate","repo":"idea-research/click-pose","repo_kind":"official","path":"datasets/coco_eval.py","file_url":"https://github.com/idea-research/click-pose/blob/HEAD/datasets/coco_eval.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fe0ddcc2d420c9a0"}},{"code_sha256_prefix":"4373b99bf6b5fd7d","entry":"ciou","repo":"idea-research/click-pose","repo_kind":"official","path":"util/box_loss.py","file_url":"https://github.com/idea-research/click-pose/blob/HEAD/util/box_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4373b99bf6b5fd7d"}},{"code_sha256_prefix":"2bcddca280475ae6","entry":"convert_coco_poly_to_mask","repo":"idea-research/click-pose","repo_kind":"official","path":"datasets/crowdpose.py","file_url":"https://github.com/idea-research/click-pose/blob/HEAD/datasets/crowdpose.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"2bcddca280475ae6"}},{"code_sha256_prefix":"7818a5040676702a","entry":"diou","repo":"idea-research/click-pose","repo_kind":"official","path":"util/box_loss.py","file_url":"https://github.com/idea-research/click-pose/blob/HEAD/util/box_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7818a5040676702a"}},{"code_sha256_prefix":"965d8927d0116879","entry":"remove","repo":"idea-research/click-pose","repo_kind":"official","path":"datasets/data_util.py","file_url":"https://github.com/idea-research/click-pose/blob/HEAD/datasets/data_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"965d8927d0116879"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}