{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/uplift-and-upsample-efficient-3d-human-pose","title":"Uplift and Upsample: Efficient 3D Human Pose Estimation with Uplifting Transformers","arxiv_id":"2210.06110","date":"2022-10-12","proceeding":null,"authors":["Moritz Einfalt","Katja Ludwig","Rainer Lienhart"],"abstract":"The state-of-the-art for monocular 3D human pose estimation in videos is dominated by the paradigm of 2D-to-3D pose uplifting. While the uplifting methods themselves are rather efficient, the true computational complexity depends on the per-frame 2D pose estimation. In this paper, we present a Transformer-based pose uplifting scheme that can operate on temporally sparse 2D pose sequences but still produce temporally dense 3D pose estimates. We show how masked token modeling can be utilized for temporal upsampling within Transformer blocks. This allows to decouple the sampling rate of input 2D poses and the target frame rate of the video and drastically decreases the total computational complexity. Additionally, we explore the option of pre-training on large motion capture archives, which has been largely neglected so far. We evaluate our method on two popular benchmark datasets: Human3.6M and MPI-INF-3DHP. With an MPJPE of 45.0 mm and 46.9 mm, respectively, our proposed method can compete with the state-of-the-art while reducing inference time by a factor of 12. This enables real-time throughput with variable consumer hardware in stationary and mobile applications. We release our code and models at https://github.com/goldbricklemon/uplift-upsample-3dhpe","url_abs":"https://arxiv.org/abs/2210.06110v3","url_pdf":"https://arxiv.org/pdf/2210.06110v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"uplift-and-upsample-efficient-3d-human-pose","repo_url":"https://github.com/goldbricklemon/uplift-upsample-3dhpe","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"uplift-and-upsample-efficient-3d-human-pose","repo_url":"https://github.com/kaulquappe23/a2b_human_mesh","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"2d-pose-estimation","task_name":"2D Pose Estimation"},{"task_slug":"3d-human-pose-estimation","task_name":"3D Human Pose Estimation"},{"task_slug":"monocular-3d-human-pose-estimation","task_name":"Monocular 3D Human Pose Estimation"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2210.06110","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.06110"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kaulquappe23/a2b_human_mesh","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/goldbricklemon/uplift-upsample-3dhpe","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":17},"by_repo_kind":{"official":{"samples":9,"ran":0,"repositories":1},"listed":{"samples":8,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"05ac12c362717f7a","entry":"create_image_paths","repo":"goldbricklemon/uplift-upsample-3dhpe","repo_kind":"official","path":"common/dataset/h36m_splits.py","file_url":"https://github.com/goldbricklemon/uplift-upsample-3dhpe/blob/HEAD/common/dataset/h36m_splits.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"05ac12c362717f7a"}},{"code_sha256_prefix":"6aafcfcb574cb205","entry":"drop_path","repo":"goldbricklemon/uplift-upsample-3dhpe","repo_kind":"official","path":"common/net/vision_transformer.py","file_url":"https://github.com/goldbricklemon/uplift-upsample-3dhpe/blob/HEAD/common/net/vision_transformer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6aafcfcb574cb205"}},{"code_sha256_prefix":"7669d6b1ce1e7096","entry":"image_coordinates","repo":"goldbricklemon/uplift-upsample-3dhpe","repo_kind":"official","path":"common/dataset/camera.py","file_url":"https://github.com/goldbricklemon/uplift-upsample-3dhpe/blob/HEAD/common/dataset/camera.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7669d6b1ce1e7096"}},{"code_sha256_prefix":"b723e9f4c150f22f","entry":"interpolate_between_keyframes","repo":"goldbricklemon/uplift-upsample-3dhpe","repo_kind":"official","path":"common/dataset/action_wise_eval.py","file_url":"https://github.com/goldbricklemon/uplift-upsample-3dhpe/blob/HEAD/common/dataset/action_wise_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b723e9f4c150f22f"}},{"code_sha256_prefix":"3b759fd80717758c","entry":"mpjpe","repo":"goldbricklemon/uplift-upsample-3dhpe","repo_kind":"official","path":"common/dataset/metrics.py","file_url":"https://github.com/goldbricklemon/uplift-upsample-3dhpe/blob/HEAD/common/dataset/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b759fd80717758c"}},{"code_sha256_prefix":"2f8afe6d7dc3c1cc","entry":"mpjpe","repo":"kaulquappe23/a2b_human_mesh","repo_kind":"listed","path":"dataset/metrics.py","file_url":"https://github.com/kaulquappe23/a2b_human_mesh/blob/HEAD/dataset/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2f8afe6d7dc3c1cc"}},{"code_sha256_prefix":"9e168cb44cb7d139","entry":"nmpjpe","repo":"goldbricklemon/uplift-upsample-3dhpe","repo_kind":"official","path":"common/dataset/metrics.py","file_url":"https://github.com/goldbricklemon/uplift-upsample-3dhpe/blob/HEAD/common/dataset/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9e168cb44cb7d139"}},{"code_sha256_prefix":"ff8248d869523ab9","entry":"nmpjpe","repo":"kaulquappe23/a2b_human_mesh","repo_kind":"listed","path":"dataset/metrics.py","file_url":"https://github.com/kaulquappe23/a2b_human_mesh/blob/HEAD/dataset/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ff8248d869523ab9"}},{"code_sha256_prefix":"82baf6aa4fb040a1","entry":"normalize_screen_coordinates","repo":"goldbricklemon/uplift-upsample-3dhpe","repo_kind":"official","path":"common/dataset/camera.py","file_url":"https://github.com/goldbricklemon/uplift-upsample-3dhpe/blob/HEAD/common/dataset/camera.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"82baf6aa4fb040a1"}},{"code_sha256_prefix":"3058f77b597a257f","entry":"np_qinverse","repo":"kaulquappe23/a2b_human_mesh","repo_kind":"listed","path":"dataset/quaternion.py","file_url":"https://github.com/kaulquappe23/a2b_human_mesh/blob/HEAD/dataset/quaternion.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3058f77b597a257f"}},{"code_sha256_prefix":"8242cae54a825bc8","entry":"np_qrot","repo":"kaulquappe23/a2b_human_mesh","repo_kind":"listed","path":"dataset/quaternion.py","file_url":"https://github.com/kaulquappe23/a2b_human_mesh/blob/HEAD/dataset/quaternion.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8242cae54a825bc8"}},{"code_sha256_prefix":"a921292bf596fdc7","entry":"pmpjpe","repo":"goldbricklemon/uplift-upsample-3dhpe","repo_kind":"official","path":"common/dataset/metrics.py","file_url":"https://github.com/goldbricklemon/uplift-upsample-3dhpe/blob/HEAD/common/dataset/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a921292bf596fdc7"}},{"code_sha256_prefix":"58f06c045c7a3705","entry":"pmpjpe","repo":"kaulquappe23/a2b_human_mesh","repo_kind":"listed","path":"dataset/metrics.py","file_url":"https://github.com/kaulquappe23/a2b_human_mesh/blob/HEAD/dataset/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"58f06c045c7a3705"}},{"code_sha256_prefix":"6a6a59635b7602ec","entry":"pt_qinverse","repo":"kaulquappe23/a2b_human_mesh","repo_kind":"listed","path":"dataset/uplifiting_dataset.py","file_url":"https://github.com/kaulquappe23/a2b_human_mesh/blob/HEAD/dataset/uplifiting_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a6a59635b7602ec"}},{"code_sha256_prefix":"95c89076de9f6be8","entry":"pt_qrot","repo":"kaulquappe23/a2b_human_mesh","repo_kind":"listed","path":"dataset/uplifiting_dataset.py","file_url":"https://github.com/kaulquappe23/a2b_human_mesh/blob/HEAD/dataset/uplifiting_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"95c89076de9f6be8"}},{"code_sha256_prefix":"eec340f4b62ea073","entry":"tf_mpjpe","repo":"goldbricklemon/uplift-upsample-3dhpe","repo_kind":"official","path":"common/utils/losses_3d.py","file_url":"https://github.com/goldbricklemon/uplift-upsample-3dhpe/blob/HEAD/common/utils/losses_3d.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eec340f4b62ea073"}},{"code_sha256_prefix":"ac0a2e613ce83f5c","entry":"world_to_cam_and_2d","repo":"kaulquappe23/a2b_human_mesh","repo_kind":"listed","path":"dataset/uplifiting_dataset.py","file_url":"https://github.com/kaulquappe23/a2b_human_mesh/blob/HEAD/dataset/uplifiting_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ac0a2e613ce83f5c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}