{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/detecting-hands-and-recognizing-physical","title":"Detecting Hands and Recognizing Physical Contact in the Wild","arxiv_id":"2010.09676","date":"2020-10-19","proceeding":"NeurIPS 2020 12","authors":["Supreeth Narasimhaswamy","Trung Nguyen","Minh Hoai"],"abstract":"We investigate a new problem of detecting hands and recognizing their physical contact state in unconstrained conditions. This is a challenging inference task given the need to reason beyond the local appearance of hands. The lack of training annotations indicating which object or parts of an object the hand is in contact with further complicates the task. We propose a novel convolutional network based on Mask-RCNN that can jointly learn to localize hands and predict their physical contact to address this problem. The network uses outputs from another object detector to obtain locations of objects present in the scene. It uses these outputs and hand locations to recognize the hand's contact state using two attention mechanisms. The first attention mechanism is based on the hand and a region's affinity, enclosing the hand and the object, and densely pools features from this region to the hand region. The second attention module adaptively selects salient features from this plausible region of contact. To develop and evaluate our method's performance, we introduce a large-scale dataset called ContactHands, containing unconstrained images annotated with hand locations and contact states. The proposed network, including the parameters of attention modules, is end-to-end trainable. This network achieves approximately 7\\% relative improvement over a baseline network that was built on the vanilla Mask-RCNN architecture and trained for recognizing hand contact states.","url_abs":"https://arxiv.org/abs/2010.09676v1","url_pdf":"https://arxiv.org/pdf/2010.09676v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"object","task_name":"Object"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2010.09676","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2010.09676"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/cvlab-stonybrook/ContactHands","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":5},"by_repo_kind":{"found_in_text":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"77c948cd3479fb3c","entry":"add_legend","repo":"cvlab-stonybrook/ContactHands","repo_kind":"found_in_text","path":"detect.py","file_url":"https://github.com/cvlab-stonybrook/ContactHands/blob/HEAD/detect.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"77c948cd3479fb3c"}},{"code_sha256_prefix":"2b7a464d9811d3ee","entry":"custom_argument_parser","repo":"cvlab-stonybrook/ContactHands","repo_kind":"found_in_text","path":"contact_hands_two_stream/engine/custom_arg_parser.py","file_url":"https://github.com/cvlab-stonybrook/ContactHands/blob/HEAD/contact_hands_two_stream/engine/custom_arg_parser.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b7a464d9811d3ee"}},{"code_sha256_prefix":"733ef63ac1f6c46e","entry":"inference_second_stream","repo":"cvlab-stonybrook/ContactHands","repo_kind":"found_in_text","path":"train_net.py","file_url":"https://github.com/cvlab-stonybrook/ContactHands/blob/HEAD/train_net.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"733ef63ac1f6c46e"}},{"code_sha256_prefix":"c510b1964cd54629","entry":"inference_second_stream","repo":"cvlab-stonybrook/ContactHands","repo_kind":"found_in_text","path":"detect.py","file_url":"https://github.com/cvlab-stonybrook/ContactHands/blob/HEAD/detect.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c510b1964cd54629"}},{"code_sha256_prefix":"611c47c44c424dba","entry":"voc_ap","repo":"cvlab-stonybrook/ContactHands","repo_kind":"found_in_text","path":"contact_hands_two_stream/evaluation/evaluator_ourdata.py","file_url":"https://github.com/cvlab-stonybrook/ContactHands/blob/HEAD/contact_hands_two_stream/evaluation/evaluator_ourdata.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"611c47c44c424dba"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}