{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cross-attention-of-disentangled-modalities","title":"Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformers","arxiv_id":"2207.13820","date":"2022-07-27","proceeding":null,"authors":["Junhyeong Cho","Kim Youwang","Tae-Hyun Oh"],"abstract":"Transformer encoder architectures have recently achieved state-of-the-art results on monocular 3D human mesh reconstruction, but they require a substantial number of parameters and expensive computations. Due to the large memory overhead and slow inference speed, it is difficult to deploy such models for practical use. In this paper, we propose a novel transformer encoder-decoder architecture for 3D human mesh reconstruction from a single image, called FastMETRO. We identify the performance bottleneck in the encoder-based transformers is caused by the token design which introduces high complexity interactions among input tokens. We disentangle the interactions via an encoder-decoder architecture, which allows our model to demand much fewer parameters and shorter inference time. In addition, we impose the prior knowledge of human body's morphological relationship via attention masking and mesh upsampling operations, which leads to faster convergence with higher accuracy. Our FastMETRO improves the Pareto-front of accuracy and efficiency, and clearly outperforms image-based methods on Human3.6M and 3DPW. Furthermore, we validate its generalizability on FreiHAND.","url_abs":"https://arxiv.org/abs/2207.13820v1","url_pdf":"https://arxiv.org/pdf/2207.13820v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cross-attention-of-disentangled-modalities","repo_url":"https://github.com/postech-ami/fastmetro","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3d-hand-pose-estimation","task_name":"3D Hand Pose Estimation"},{"task_slug":"3d-human-pose-estimation","task_name":"3D Human Pose Estimation"},{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"},{"task_slug":"decoder","task_name":"Decoder"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-hand-pose-estimation-on-freihand","task":"3D Hand Pose Estimation","dataset":"FreiHAND","model":"FastMETRO","rank_in_archive_order":16,"of":33,"metrics":{"PA-F@15mm":"0.983","PA-F@5mm":"0.687","PA-MPJPE":"6.5","PA-MPVPE":"7.1"},"uses_additional_data":false},{"leaderboard":"/sota/3d-human-pose-estimation-on-emdb","task":"3D Human Pose Estimation","dataset":"EMDB","model":"FastMETRO-L no SMPL Head","rank_in_archive_order":6,"of":13,"metrics":{"Average MPJPE (mm)":"108.107","Average MPJPE-PA (mm)":"66.794","Average MVE (mm)":"119.23","Average MVE-PA (mm)":"81.1847","Jitter (10m/s^3)":"185.933"},"uses_additional_data":false},{"leaderboard":"/sota/3d-human-pose-estimation-on-emdb","task":"3D Human Pose Estimation","dataset":"EMDB","model":"FastMETRO-L","rank_in_archive_order":10,"of":13,"metrics":{"Average MPJAE (deg)":"25.07","Average MPJAE-PA (deg)":"22.9482","Average MPJPE (mm)":"115.036","Average MPJPE-PA (mm)":"72.6765","Average MVE (mm)":"133.566","Average MVE-PA (mm)":"86.0043","Jitter (10m/s^3)":"81.2959"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2207.13820","atlas_url":"https://app.syntology.ai/?focus=2207.13820","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.13820"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/postech-ami/fastmetro","reach":null}],"summary":{"ran":8,"ran_draft_wrong":3,"ran_fixture":1,"unverified":4},"by_repo_kind":{"official":{"samples":16,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"562c62242b4e4a11","entry":"FCBlock","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"562c62242b4e4a11"}},{"code_sha256_prefix":"4d8d11aa70b28920","entry":"FCResBlock","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4d8d11aa70b28920"}},{"code_sha256_prefix":"ab7c9f19268bf5b1","entry":"PositionEmbeddingSine","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ab7c9f19268bf5b1"}},{"code_sha256_prefix":"4a7551bd532ddd12","entry":"TransformerDecoder","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4a7551bd532ddd12"}},{"code_sha256_prefix":"b7f02dc052c23ed5","entry":"TransformerDecoderLayer","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b7f02dc052c23ed5"}},{"code_sha256_prefix":"f432265aed9e2e0b","entry":"TransformerEncoder","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f432265aed9e2e0b"}},{"code_sha256_prefix":"81cad16cd0d5efb1","entry":"TransformerEncoderLayer","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"81cad16cd0d5efb1"}},{"code_sha256_prefix":"d493e14665ada3f2","entry":"_get_activation_fn","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d493e14665ada3f2"}},{"code_sha256_prefix":"873e680b6cfcd7a5","entry":"_get_clones","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"873e680b6cfcd7a5"}},{"code_sha256_prefix":"6697b1ba4e63ee2a","entry":"batch_svd","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6697b1ba4e63ee2a"}},{"code_sha256_prefix":"4407bd06305a2f6c","entry":"build_position_encoding","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4407bd06305a2f6c"}},{"code_sha256_prefix":"52fdd40a81f5afe0","entry":"build_smpl_parameter_regressor","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"52fdd40a81f5afe0"}},{"code_sha256_prefix":"f8c49af84f39dc14","entry":"FastMETRO_Body_Network","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f8c49af84f39dc14"}},{"code_sha256_prefix":"12981f7af75b69b4","entry":"SMPL_Parameter_Regressor","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"12981f7af75b69b4"}},{"code_sha256_prefix":"bc515da1433c761e","entry":"Transformer","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc515da1433c761e"}},{"code_sha256_prefix":"c6caa9a5fb3f06a7","entry":"build_transformer","repo":"postech-ami/fastmetro","repo_kind":"official","path":"src/modeling/model/modeling_fastmetro.py","file_url":"https://github.com/postech-ami/fastmetro/blob/HEAD/src/modeling/model/modeling_fastmetro.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c6caa9a5fb3f06a7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}