{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/p-stmo-pre-trained-spatial-temporal-many-to","title":"P-STMO: Pre-Trained Spatial Temporal Many-to-One Model for 3D Human Pose Estimation","arxiv_id":"2203.07628","date":"2022-03-15","proceeding":null,"authors":["Wenkang Shan","Zhenhua Liu","Xinfeng Zhang","Shanshe Wang","Siwei Ma","Wen Gao"],"abstract":"This paper introduces a novel Pre-trained Spatial Temporal Many-to-One (P-STMO) model for 2D-to-3D human pose estimation task. To reduce the difficulty of capturing spatial and temporal information, we divide this task into two stages: pre-training (Stage I) and fine-tuning (Stage II). In Stage I, a self-supervised pre-training sub-task, termed masked pose modeling, is proposed. The human joints in the input sequence are randomly masked in both spatial and temporal domains. A general form of denoising auto-encoder is exploited to recover the original 2D poses and the encoder is capable of capturing spatial and temporal dependencies in this way. In Stage II, the pre-trained encoder is loaded to STMO model and fine-tuned. The encoder is followed by a many-to-one frame aggregator to predict the 3D pose in the current frame. Especially, an MLP block is utilized as the spatial feature extractor in STMO, which yields better performance than other methods. In addition, a temporal downsampling strategy is proposed to diminish data redundancy. Extensive experiments on two benchmarks show that our method outperforms state-of-the-art methods with fewer parameters and less computational overhead. For example, our P-STMO model achieves 42.1mm MPJPE on Human3.6M dataset when using 2D poses from CPN as inputs. Meanwhile, it brings a 1.5-7.1 times speedup to state-of-the-art methods. Code is available at https://github.com/paTRICK-swk/P-STMO.","url_abs":"https://arxiv.org/abs/2203.07628v2","url_pdf":"https://arxiv.org/pdf/2203.07628v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"p-stmo-pre-trained-spatial-temporal-many-to","repo_url":"https://github.com/patrick-swk/p-stmo","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"3d-human-pose-estimation","task_name":"3D Human Pose Estimation"},{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"monocular-3d-human-pose-estimation","task_name":"Monocular 3D Human Pose Estimation"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"}],"methods":[{"method_slug":"cpn","method_name":"CPN"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"non-maximum-suppression","method_name":"Non Maximum Suppression"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-human-pose-estimation-on-human36m","task":"3D Human Pose Estimation","dataset":"Human3.6M","model":"P-STMO (N=243)","rank_in_archive_order":23,"of":88,"metrics":{"Average MPJPE (mm)":"42.1","Multi-View or Monocular":"Monocular","PA-MPJPE":"34.4","Using 2D ground-truth joints":"No"},"uses_additional_data":false},{"leaderboard":"/sota/3d-human-pose-estimation-on-human36m","task":"3D Human Pose Estimation","dataset":"Human3.6M","model":"P-STMO-S (N=81)","rank_in_archive_order":34,"of":88,"metrics":{"Average MPJPE (mm)":"44.1","Multi-View or Monocular":"Monocular","Using 2D ground-truth joints":"No"},"uses_additional_data":false},{"leaderboard":"/sota/3d-human-pose-estimation-on-mpi-inf-3dhp","task":"3D Human Pose Estimation","dataset":"MPI-INF-3DHP","model":"P-STMO (N=81)","rank_in_archive_order":15,"of":108,"metrics":{"AUC":"75.8","MPJPE":"32.2","PCK":"97.9"},"uses_additional_data":false},{"leaderboard":"/sota/monocular-3d-human-pose-estimation-on-human3","task":"Monocular 3D Human Pose Estimation","dataset":"Human3.6M","model":"P-STMO (N=243)","rank_in_archive_order":12,"of":52,"metrics":{"2D detector":"CPN","Average MPJPE (mm)":"42.1","Frames Needed":"243","Need Ground Truth 2D Pose":"No","Use Video Sequence":"Yes"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.07628","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.07628"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/patrick-swk/p-stmo","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/paTRICK-swk/P-STMO","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":7,"ran_draft_wrong":2,"unverified":2},"by_repo_kind":{"official":{"samples":10,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"1aaaa9db258f9346","entry":"Encoder","repo":"paTRICK-swk/P-STMO","repo_kind":"official","path":"model/stmo_pretrain.py","file_url":"https://github.com/paTRICK-swk/P-STMO/blob/HEAD/model/stmo_pretrain.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1aaaa9db258f9346"}},{"code_sha256_prefix":"1e31c11931ce7561","entry":"EncoderLayer","repo":"paTRICK-swk/P-STMO","repo_kind":"official","path":"model/stmo_pretrain.py","file_url":"https://github.com/paTRICK-swk/P-STMO/blob/HEAD/model/stmo_pretrain.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1e31c11931ce7561"}},{"code_sha256_prefix":"3dd75f638909b681","entry":"FCBlock","repo":"paTRICK-swk/P-STMO","repo_kind":"official","path":"model/stmo_pretrain.py","file_url":"https://github.com/paTRICK-swk/P-STMO/blob/HEAD/model/stmo_pretrain.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3dd75f638909b681"}},{"code_sha256_prefix":"106e2299d93b5ed7","entry":"Linear","repo":"paTRICK-swk/P-STMO","repo_kind":"official","path":"model/stmo_pretrain.py","file_url":"https://github.com/paTRICK-swk/P-STMO/blob/HEAD/model/stmo_pretrain.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"106e2299d93b5ed7"}},{"code_sha256_prefix":"9bc6abc38072722f","entry":"MultiHeadedAttention","repo":"paTRICK-swk/P-STMO","repo_kind":"official","path":"model/stmo_pretrain.py","file_url":"https://github.com/paTRICK-swk/P-STMO/blob/HEAD/model/stmo_pretrain.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9bc6abc38072722f"}},{"code_sha256_prefix":"cb80183aeec842d5","entry":"PositionwiseFeedForward","repo":"paTRICK-swk/P-STMO","repo_kind":"official","path":"model/stmo_pretrain.py","file_url":"https://github.com/paTRICK-swk/P-STMO/blob/HEAD/model/stmo_pretrain.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb80183aeec842d5"}},{"code_sha256_prefix":"56c8855e69bdbfe5","entry":"Transformer","repo":"paTRICK-swk/P-STMO","repo_kind":"official","path":"model/stmo_pretrain.py","file_url":"https://github.com/paTRICK-swk/P-STMO/blob/HEAD/model/stmo_pretrain.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"56c8855e69bdbfe5"}},{"code_sha256_prefix":"cbe07626e47b77c3","entry":"attention","repo":"paTRICK-swk/P-STMO","repo_kind":"official","path":"model/stmo_pretrain.py","file_url":"https://github.com/paTRICK-swk/P-STMO/blob/HEAD/model/stmo_pretrain.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cbe07626e47b77c3"}},{"code_sha256_prefix":"d2d810cddae9f875","entry":"clones","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"d2d810cddae9f875"}},{"code_sha256_prefix":"75538e6a2c58ef23","entry":"Model_MAE","repo":"paTRICK-swk/P-STMO","repo_kind":"official","path":"model/stmo_pretrain.py","file_url":"https://github.com/paTRICK-swk/P-STMO/blob/HEAD/model/stmo_pretrain.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"75538e6a2c58ef23"}},{"code_sha256_prefix":"a135de1a158f5ac6","entry":"Transformer_dec","repo":"paTRICK-swk/P-STMO","repo_kind":"official","path":"model/stmo_pretrain.py","file_url":"https://github.com/paTRICK-swk/P-STMO/blob/HEAD/model/stmo_pretrain.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a135de1a158f5ac6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}