{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/masked-motion-predictors-are-strong-3d-action","title":"Masked Motion Predictors are Strong 3D Action Representation Learners","arxiv_id":"2308.07092","date":"2023-08-14","proceeding":"ICCV 2023 1","authors":["Yunyao Mao","Jiajun Deng","Wengang Zhou","Yao Fang","Wanli Ouyang","Houqiang Li"],"abstract":"In 3D human action recognition, limited supervised data makes it challenging to fully tap into the modeling potential of powerful networks such as transformers. As a result, researchers have been actively investigating effective self-supervised pre-training strategies. In this work, we show that instead of following the prevalent pretext task to perform masked self-component reconstruction in human joints, explicit contextual motion modeling is key to the success of learning effective feature representation for 3D action recognition. Formally, we propose the Masked Motion Prediction (MAMP) framework. To be specific, the proposed MAMP takes as input the masked spatio-temporal skeleton sequence and predicts the corresponding temporal motion of the masked human joints. Considering the high temporal redundancy of the skeleton sequence, in our MAMP, the motion information also acts as an empirical semantic richness prior that guide the masking process, promoting better attention to semantically rich temporal regions. Extensive experiments on NTU-60, NTU-120, and PKU-MMD datasets show that the proposed MAMP pre-training substantially improves the performance of the adopted vanilla transformer, achieving state-of-the-art results without bells and whistles. The source code of our MAMP is available at https://github.com/maoyunyao/MAMP.","url_abs":"https://arxiv.org/abs/2308.07092v1","url_pdf":"https://arxiv.org/pdf/2308.07092v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"masked-motion-predictors-are-strong-3d-action","repo_url":"https://github.com/maoyunyao/mamp","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"3d-human-action-recognition","task_name":"3D Action Recognition"},{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"few-shot-skeleton-based-action-recognition","task_name":"Few-Shot Skeleton-Based Action Recognition"},{"task_slug":"self-supervised-skeleton-based-action","task_name":"Self-supervised Skeleton-based Action Recognition"},{"task_slug":"skeleton-based-action-recognition","task_name":"Skeleton Based Action Recognition"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"},{"task_slug":"motion-prediction","task_name":"motion prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/skeleton-based-action-recognition-on-ntu-rgbd","task":"Skeleton Based Action Recognition","dataset":"NTU RGB+D","model":"MAMP","rank_in_archive_order":18,"of":135,"metrics":{"Accuracy (CS)":"93.1","Accuracy (CV)":"97.5"},"uses_additional_data":false},{"leaderboard":"/sota/skeleton-based-action-recognition-on-ntu-rgbd-1","task":"Skeleton Based Action Recognition","dataset":"NTU RGB+D 120","model":"MAMP","rank_in_archive_order":10,"of":83,"metrics":{"Accuracy (Cross-Setup)":"91.3","Accuracy (Cross-Subject)":"90.0"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2308.07092","atlas_url":"https://app.syntology.ai/?focus=2308.07092","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.07092"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/maoyunyao/mamp","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/maoyunyao/MAMP","reach":{"status":"ok"}}],"summary":{"ran_fixture":2,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"3ac6b7d76e8e3584","entry":"drop_path","repo":"maoyunyao/mamp","repo_kind":"official","path":"model_mamp/drop.py","file_url":"https://github.com/maoyunyao/mamp/blob/HEAD/model_mamp/drop.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3ac6b7d76e8e3584"}},{"code_sha256_prefix":"ffed4f85d50832c9","entry":"import_class","repo":"maoyunyao/mamp","repo_kind":"official","path":"main_finetune.py","file_url":"https://github.com/maoyunyao/mamp/blob/HEAD/main_finetune.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ffed4f85d50832c9"}},{"code_sha256_prefix":"02566da69866c48c","entry":"trunc_normal_","repo":"maoyunyao/mamp","repo_kind":"official","path":"model_mamp/transformer.py","file_url":"https://github.com/maoyunyao/mamp/blob/HEAD/model_mamp/transformer.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"02566da69866c48c"}},{"code_sha256_prefix":"37f8888ae9c0b863","entry":"Transformer","repo":"maoyunyao/MAMP","repo_kind":"official","path":"model_mamp/transformer.py","file_url":"https://github.com/maoyunyao/MAMP/blob/HEAD/model_mamp/transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"37f8888ae9c0b863"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}