{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ktpformer-kinematics-and-trajectory-prior","title":"KTPFormer: Kinematics and Trajectory Prior Knowledge-Enhanced Transformer for 3D Human Pose Estimation","arxiv_id":"2404.00658","date":"2024-03-31","proceeding":"CVPR 2024 1","authors":["Jihua Peng","Yanghong Zhou","P. Y. Mok"],"abstract":"This paper presents a novel Kinematics and Trajectory Prior Knowledge-Enhanced Transformer (KTPFormer), which overcomes the weakness in existing transformer-based methods for 3D human pose estimation that the derivation of Q, K, V vectors in their self-attention mechanisms are all based on simple linear mapping. We propose two prior attention modules, namely Kinematics Prior Attention (KPA) and Trajectory Prior Attention (TPA) to take advantage of the known anatomical structure of the human body and motion trajectory information, to facilitate effective learning of global dependencies and features in the multi-head self-attention. KPA models kinematic relationships in the human body by constructing a topology of kinematics, while TPA builds a trajectory topology to learn the information of joint motion trajectory across frames. Yielding Q, K, V vectors with prior knowledge, the two modules enable KTPFormer to model both spatial and temporal correlations simultaneously. Extensive experiments on three benchmarks (Human3.6M, MPI-INF-3DHP and HumanEva) show that KTPFormer achieves superior performance in comparison to state-of-the-art methods. More importantly, our KPA and TPA modules have lightweight plug-and-play designs and can be integrated into various transformer-based networks (i.e., diffusion-based) to improve the performance with only a very small increase in the computational overhead. The code is available at: https://github.com/JihuaPeng/KTPFormer.","url_abs":"https://arxiv.org/abs/2404.00658v2","url_pdf":"https://arxiv.org/pdf/2404.00658v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ktpformer-kinematics-and-trajectory-prior","repo_url":"https://github.com/JihuaPeng/KTPFormer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"3d-human-pose-estimation","task_name":"3D Human Pose Estimation"},{"task_slug":"monocular-3d-human-pose-estimation","task_name":"Monocular 3D Human Pose Estimation"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-human-pose-estimation-on-human36m","task":"3D Human Pose Estimation","dataset":"Human3.6M","model":"KTPFormer (T=243)","rank_in_archive_order":12,"of":88,"metrics":{"Average MPJPE (mm)":"33.0","Multi-View or Monocular":"Monocular","PA-MPJPE":"26.2","Using 2D ground-truth joints":"No"},"uses_additional_data":false},{"leaderboard":"/sota/3d-human-pose-estimation-on-mpi-inf-3dhp","task":"3D Human Pose Estimation","dataset":"MPI-INF-3DHP","model":"KTPFormer","rank_in_archive_order":3,"of":108,"metrics":{"AUC":"85.9","MPJPE":"16.7","PCK":"98.9"},"uses_additional_data":false},{"leaderboard":"/sota/monocular-3d-human-pose-estimation-on-human3","task":"Monocular 3D Human Pose Estimation","dataset":"Human3.6M","model":"KTPFormer","rank_in_archive_order":10,"of":52,"metrics":{"2D detector":"CPN","Average MPJPE (mm)":"40.1","Frames Needed":"243","Need Ground Truth 2D Pose":"No","Use Video Sequence":"Yes"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2404.00658","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.00658"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/JihuaPeng/KTPFormer","reach":{"status":"ok"}}],"summary":{"ran":4,"ran_violates":1,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":10,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"8e11f61858f90f83","entry":"mpjpe","repo":"JihuaPeng/KTPFormer","repo_kind":"official","path":"common/loss.py","file_url":"https://github.com/JihuaPeng/KTPFormer/blob/HEAD/common/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8e11f61858f90f83"}},{"code_sha256_prefix":"aa9c29936dee40a7","entry":"normalize","repo":"JihuaPeng/KTPFormer","repo_kind":"official","path":"common/graph_utils.py","file_url":"https://github.com/JihuaPeng/KTPFormer/blob/HEAD/common/graph_utils.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa9c29936dee40a7"}},{"code_sha256_prefix":"29d56deadc8fcd3c","entry":"p_mpjpe","repo":"JihuaPeng/KTPFormer","repo_kind":"official","path":"common/loss.py","file_url":"https://github.com/JihuaPeng/KTPFormer/blob/HEAD/common/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"29d56deadc8fcd3c"}},{"code_sha256_prefix":"6cc69ac50608b9b0","entry":"qinverse","repo":"JihuaPeng/KTPFormer","repo_kind":"official","path":"common/quaternion.py","file_url":"https://github.com/JihuaPeng/KTPFormer/blob/HEAD/common/quaternion.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6cc69ac50608b9b0"}},{"code_sha256_prefix":"34067eef72357f72","entry":"qrot","repo":"JihuaPeng/KTPFormer","repo_kind":"official","path":"common/quaternion.py","file_url":"https://github.com/JihuaPeng/KTPFormer/blob/HEAD/common/quaternion.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"34067eef72357f72"}},{"code_sha256_prefix":"bdc1f1dad6d978e7","entry":"sparse_mx_to_torch_sparse_tensor","repo":"JihuaPeng/KTPFormer","repo_kind":"official","path":"common/graph_utils.py","file_url":"https://github.com/JihuaPeng/KTPFormer/blob/HEAD/common/graph_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bdc1f1dad6d978e7"}},{"code_sha256_prefix":"8080a0e016046baf","entry":"adj_mx_from_edges","repo":"JihuaPeng/KTPFormer","repo_kind":"official","path":"common/graph_utils.py","file_url":"https://github.com/JihuaPeng/KTPFormer/blob/HEAD/common/graph_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8080a0e016046baf"}},{"code_sha256_prefix":"7669d6b1ce1e7096","entry":"image_coordinates","repo":"JihuaPeng/KTPFormer","repo_kind":"official","path":"common/camera.py","file_url":"https://github.com/JihuaPeng/KTPFormer/blob/HEAD/common/camera.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7669d6b1ce1e7096"}},{"code_sha256_prefix":"82baf6aa4fb040a1","entry":"normalize_screen_coordinates","repo":"JihuaPeng/KTPFormer","repo_kind":"official","path":"common/camera.py","file_url":"https://github.com/JihuaPeng/KTPFormer/blob/HEAD/common/camera.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"82baf6aa4fb040a1"}},{"code_sha256_prefix":"09a4e544aff13453","entry":"weighted_mpjpe","repo":"JihuaPeng/KTPFormer","repo_kind":"official","path":"common/loss.py","file_url":"https://github.com/JihuaPeng/KTPFormer/blob/HEAD/common/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"09a4e544aff13453"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}