{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hap-structure-aware-masked-image-modeling-for-1","title":"HAP: Structure-Aware Masked Image Modeling for Human-Centric Perception","arxiv_id":"2310.20695","date":"2023-10-31","proceeding":"NeurIPS 2023 11","authors":["Junkun Yuan","Xinyu Zhang","Hao Zhou","Jian Wang","Zhongwei Qiu","Zhiyin Shao","Shaofeng Zhang","Sifan Long","Kun Kuang","Kun Yao","Junyu Han","Errui Ding","Lanfen Lin","Fei Wu","Jingdong Wang"],"abstract":"Model pre-training is essential in human-centric perception. In this paper, we first introduce masked image modeling (MIM) as a pre-training approach for this task. Upon revisiting the MIM training strategy, we reveal that human structure priors offer significant potential. Motivated by this insight, we further incorporate an intuitive human structure prior - human parts - into pre-training. Specifically, we employ this prior to guide the mask sampling process. Image patches, corresponding to human part regions, have high priority to be masked out. This encourages the model to concentrate more on body structure information during pre-training, yielding substantial benefits across a range of human-centric perception tasks. To further capture human characteristics, we propose a structure-invariant alignment loss that enforces different masked views, guided by the human part prior, to be closely aligned for the same image. We term the entire method as HAP. HAP simply uses a plain ViT as the encoder yet establishes new state-of-the-art performance on 11 human-centric benchmarks, and on-par result on one dataset. For example, HAP achieves 78.1% mAP on MSMT17 for person re-identification, 86.54% mA on PA-100K for pedestrian attribute recognition, 78.2% AP on MS COCO for 2D pose estimation, and 56.0 PA-MPJPE on 3DPW for 3D pose and shape estimation.","url_abs":"https://arxiv.org/abs/2310.20695v1","url_pdf":"https://arxiv.org/pdf/2310.20695v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hap-structure-aware-masked-image-modeling-for-1","repo_url":"https://github.com/junkunyuan/hap","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"2d-pose-estimation","task_name":"2D Pose Estimation"},{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"pedestrian-attribute-recognition","task_name":"Pedestrian Attribute Recognition"},{"task_slug":"person-re-identification","task_name":"Person Re-Identification"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"}],"methods":[{"method_slug":"mim","method_name":"MIM"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.20695","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.20695"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/junkunyuan/hap","reach":null}],"summary":{"ran":1,"ran_draft_wrong":2,"unverified":3},"by_repo_kind":{"official":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"e31a5c9ec6aace50","entry":"MaskingGenerator","repo":"junkunyuan/hap","repo_kind":"official","path":"models/models_mae_pose.py","file_url":"https://github.com/junkunyuan/hap/blob/HEAD/models/models_mae_pose.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e31a5c9ec6aace50"}},{"code_sha256_prefix":"1cae3dc8b2456c8e","entry":"get_patch","repo":"junkunyuan/hap","repo_kind":"official","path":"models/models_mae_pose.py","file_url":"https://github.com/junkunyuan/hap/blob/HEAD/models/models_mae_pose.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1cae3dc8b2456c8e"}},{"code_sha256_prefix":"f78e7481d17e98bb","entry":"mask_body_parts","repo":"junkunyuan/hap","repo_kind":"official","path":"models/models_mae_pose.py","file_url":"https://github.com/junkunyuan/hap/blob/HEAD/models/models_mae_pose.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f78e7481d17e98bb"}},{"code_sha256_prefix":"3945f63a6f011f34","entry":"AttentionAtt","repo":"junkunyuan/hap","repo_kind":"official","path":"models/models_mae_pose.py","file_url":"https://github.com/junkunyuan/hap/blob/HEAD/models/models_mae_pose.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3945f63a6f011f34"}},{"code_sha256_prefix":"f1e9b4d49560e00a","entry":"BlockAtt","repo":"junkunyuan/hap","repo_kind":"official","path":"models/models_mae_pose.py","file_url":"https://github.com/junkunyuan/hap/blob/HEAD/models/models_mae_pose.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f1e9b4d49560e00a"}},{"code_sha256_prefix":"04a7050cba855969","entry":"PoseMaskedAutoencoderViT","repo":"junkunyuan/hap","repo_kind":"official","path":"models/models_mae_pose.py","file_url":"https://github.com/junkunyuan/hap/blob/HEAD/models/models_mae_pose.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"04a7050cba855969"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}