{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/auxiliary-tasks-benefit-3d-skeleton-based","title":"Auxiliary Tasks Benefit 3D Skeleton-based Human Motion Prediction","arxiv_id":"2308.08942","date":"2023-08-17","proceeding":"ICCV 2023 1","authors":["Chenxin Xu","Robby T. Tan","Yuhong Tan","Siheng Chen","Xinchao Wang","Yanfeng Wang"],"abstract":"Exploring spatial-temporal dependencies from observed motions is one of the core challenges of human motion prediction. Previous methods mainly focus on dedicated network structures to model the spatial and temporal dependencies. This paper considers a new direction by introducing a model learning framework with auxiliary tasks. In our auxiliary tasks, partial body joints' coordinates are corrupted by either masking or adding noise and the goal is to recover corrupted coordinates depending on the rest coordinates. To work with auxiliary tasks, we propose a novel auxiliary-adapted transformer, which can handle incomplete, corrupted motion data and achieve coordinate recovery via capturing spatial-temporal dependencies. Through auxiliary tasks, the auxiliary-adapted transformer is promoted to capture more comprehensive spatial-temporal dependencies among body joints' coordinates, leading to better feature learning. Extensive experimental results have shown that our method outperforms state-of-the-art methods by remarkable margins of 7.2%, 3.7%, and 9.4% in terms of 3D mean per joint position error (MPJPE) on the Human3.6M, CMU Mocap, and 3DPW datasets, respectively. We also demonstrate that our method is more robust under data missing cases and noisy data cases. Code is available at https://github.com/MediaBrain-SJTU/AuxFormer.","url_abs":"https://arxiv.org/abs/2308.08942v1","url_pdf":"https://arxiv.org/pdf/2308.08942v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"auxiliary-tasks-benefit-3d-skeleton-based","repo_url":"https://github.com/mediabrain-sjtu/auxformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"human-pose-forecasting","task_name":"Human Pose Forecasting"},{"task_slug":"human-motion-prediction","task_name":"Human motion prediction"},{"task_slug":"motion-prediction","task_name":"motion prediction"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/human-pose-forecasting-on-3dpw","task":"Human Pose Forecasting","dataset":"3DPW","model":"AuxFormer","rank_in_archive_order":6,"of":7,"metrics":{"Average MPJPE (mm) 1000 msec":"107.45"},"uses_additional_data":false},{"leaderboard":"/sota/human-pose-forecasting-on-human36m","task":"Human Pose Forecasting","dataset":"Human3.6M","model":"AuxFormer","rank_in_archive_order":4,"of":33,"metrics":{"Average MPJPE (mm) @ 1000 ms":"107","Average MPJPE (mm) @ 400ms":"54.1"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2308.08942","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.08942"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mediabrain-sjtu/auxformer","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/MediaBrain-SJTU/AuxFormer","reach":{"status":"ok"}}],"summary":{"ran":12,"ran_honours":1,"unverified":6},"by_repo_kind":{"official":{"samples":19,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":19,"samples":[{"code_sha256_prefix":"692b43d6b0a5d50a","entry":"FFN","repo":"MediaBrain-SJTU/AuxFormer","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/MediaBrain-SJTU/AuxFormer/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"692b43d6b0a5d50a"}},{"code_sha256_prefix":"3c0bd7c5cee04dff","entry":"PreNorm","repo":"MediaBrain-SJTU/AuxFormer","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/MediaBrain-SJTU/AuxFormer/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3c0bd7c5cee04dff"}},{"code_sha256_prefix":"3632680f7c06faf1","entry":"STTrans","repo":"MediaBrain-SJTU/AuxFormer","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/MediaBrain-SJTU/AuxFormer/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3632680f7c06faf1"}},{"code_sha256_prefix":"39d376502182d85a","entry":"SelfAttention","repo":"MediaBrain-SJTU/AuxFormer","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/MediaBrain-SJTU/AuxFormer/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"39d376502182d85a"}},{"code_sha256_prefix":"13c3d291513f3615","entry":"Transformer","repo":"MediaBrain-SJTU/AuxFormer","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/MediaBrain-SJTU/AuxFormer/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"13c3d291513f3615"}},{"code_sha256_prefix":"a45512bd382da8b8","entry":"define_actions","repo":"mediabrain-sjtu/auxformer","repo_kind":"official","path":"dataset/data_tools.py","file_url":"https://github.com/mediabrain-sjtu/auxformer/blob/HEAD/dataset/data_tools.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a45512bd382da8b8"}},{"code_sha256_prefix":"5bb58894a342db5d","entry":"load_data","repo":"mediabrain-sjtu/auxformer","repo_kind":"official","path":"dataset/data_tools.py","file_url":"https://github.com/mediabrain-sjtu/auxformer/blob/HEAD/dataset/data_tools.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5bb58894a342db5d"}},{"code_sha256_prefix":"d967154c60f2573c","entry":"lr_decay","repo":"mediabrain-sjtu/auxformer","repo_kind":"official","path":"main_3dpw.py","file_url":"https://github.com/mediabrain-sjtu/auxformer/blob/HEAD/main_3dpw.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d967154c60f2573c"}},{"code_sha256_prefix":"238bc05bde05cf2f","entry":"read_txt_as_data","repo":"mediabrain-sjtu/auxformer","repo_kind":"official","path":"dataset/data_tools.py","file_url":"https://github.com/mediabrain-sjtu/auxformer/blob/HEAD/dataset/data_tools.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"238bc05bde05cf2f"}},{"code_sha256_prefix":"3fd47e4842ab099f","entry":"rotmat2euler","repo":"mediabrain-sjtu/auxformer","repo_kind":"official","path":"dataset/data_utils.py","file_url":"https://github.com/mediabrain-sjtu/auxformer/blob/HEAD/dataset/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3fd47e4842ab099f"}},{"code_sha256_prefix":"7091114ed708fb93","entry":"rotmat2expmap","repo":"mediabrain-sjtu/auxformer","repo_kind":"official","path":"dataset/data_utils.py","file_url":"https://github.com/mediabrain-sjtu/auxformer/blob/HEAD/dataset/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7091114ed708fb93"}},{"code_sha256_prefix":"2a3ccebf0173ab84","entry":"rotmat2quat","repo":"mediabrain-sjtu/auxformer","repo_kind":"official","path":"dataset/data_utils.py","file_url":"https://github.com/mediabrain-sjtu/auxformer/blob/HEAD/dataset/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2a3ccebf0173ab84"}},{"code_sha256_prefix":"35407e9c8621ad92","entry":"to_pair","repo":"mediabrain-sjtu/auxformer","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/mediabrain-sjtu/auxformer/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"35407e9c8621ad92"}},{"code_sha256_prefix":"41656154a7eeff9b","entry":"AuxFormer","repo":"MediaBrain-SJTU/AuxFormer","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/MediaBrain-SJTU/AuxFormer/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"41656154a7eeff9b"}},{"code_sha256_prefix":"2bf3431aa40754e1","entry":"test","repo":"mediabrain-sjtu/auxformer","repo_kind":"official","path":"main_3dpw.py","file_url":"https://github.com/mediabrain-sjtu/auxformer/blob/HEAD/main_3dpw.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2bf3431aa40754e1"}},{"code_sha256_prefix":"138adfd1f5f6387c","entry":"test","repo":"mediabrain-sjtu/auxformer","repo_kind":"official","path":"main_cmu.py","file_url":"https://github.com/mediabrain-sjtu/auxformer/blob/HEAD/main_cmu.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"138adfd1f5f6387c"}},{"code_sha256_prefix":"6a792b53215ecac8","entry":"test","repo":"mediabrain-sjtu/auxformer","repo_kind":"official","path":"main_h36m.py","file_url":"https://github.com/mediabrain-sjtu/auxformer/blob/HEAD/main_h36m.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6a792b53215ecac8"}},{"code_sha256_prefix":"77377e7ece06c283","entry":"train","repo":"mediabrain-sjtu/auxformer","repo_kind":"official","path":"main_3dpw.py","file_url":"https://github.com/mediabrain-sjtu/auxformer/blob/HEAD/main_3dpw.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"77377e7ece06c283"}},{"code_sha256_prefix":"2b8d99de10074ebc","entry":"train","repo":"mediabrain-sjtu/auxformer","repo_kind":"official","path":"main_cmu.py","file_url":"https://github.com/mediabrain-sjtu/auxformer/blob/HEAD/main_cmu.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2b8d99de10074ebc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}