{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/jotr-3d-joint-contrastive-learning-with","title":"JOTR: 3D Joint Contrastive Learning with Transformers for Occluded Human Mesh Recovery","arxiv_id":"2307.16377","date":"2023-07-31","proceeding":"ICCV 2023 1","authors":["Jiahao Li","Zongxin Yang","Xiaohan Wang","Jianxin Ma","Chang Zhou","Yi Yang"],"abstract":"In this study, we focus on the problem of 3D human mesh recovery from a single image under obscured conditions. Most state-of-the-art methods aim to improve 2D alignment technologies, such as spatial averaging and 2D joint sampling. However, they tend to neglect the crucial aspect of 3D alignment by improving 3D representations. Furthermore, recent methods struggle to separate the target human from occlusion or background in crowded scenes as they optimize the 3D space of target human with 3D joint coordinates as local supervision. To address these issues, a desirable method would involve a framework for fusing 2D and 3D features and a strategy for optimizing the 3D space globally. Therefore, this paper presents 3D JOint contrastive learning with TRansformers (JOTR) framework for handling occluded 3D human mesh recovery. Our method includes an encoder-decoder transformer architecture to fuse 2D and 3D representations for achieving 2D$\\&$3D aligned results in a coarse-to-fine manner and a novel 3D joint contrastive learning approach for adding explicitly global supervision for the 3D feature space. The contrastive learning approach includes two contrastive losses: joint-to-joint contrast for enhancing the similarity of semantically similar voxels (i.e., human joints), and joint-to-non-joint contrast for ensuring discrimination from others (e.g., occlusions and background). Qualitative and quantitative analyses demonstrate that our method outperforms state-of-the-art competitors on both occlusion-specific and standard benchmarks, significantly improving the reconstruction of occluded humans.","url_abs":"https://arxiv.org/abs/2307.16377v2","url_pdf":"https://arxiv.org/pdf/2307.16377v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"jotr-3d-joint-contrastive-learning-with","repo_url":"https://github.com/xljh0520/jotr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"human-mesh-recovery","task_name":"Human Mesh Recovery"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"},{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2307.16377","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.16377"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xljh0520/jotr","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/xljh0520/JOTR","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":7,"ran_draft_wrong":3,"ran_violates":1,"unverified":3},"by_repo_kind":{"official":{"samples":14,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2a17d62dc182d895","entry":"TransformerDecoder","repo":"xljh0520/JOTR","repo_kind":"official","path":"common/nets/transformer.py","file_url":"https://github.com/xljh0520/JOTR/blob/HEAD/common/nets/transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a17d62dc182d895"}},{"code_sha256_prefix":"0f4367575569ca12","entry":"bulid_transformer_encoder","repo":"xljh0520/jotr","repo_kind":"official","path":"common/nets/transformer.py","file_url":"https://github.com/xljh0520/jotr/blob/HEAD/common/nets/transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f4367575569ca12"}},{"code_sha256_prefix":"d9def42110729a85","entry":"conv1x1","repo":"xljh0520/jotr","repo_kind":"official","path":"common/nets/resnet.py","file_url":"https://github.com/xljh0520/jotr/blob/HEAD/common/nets/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9def42110729a85"}},{"code_sha256_prefix":"160bb14bd76201b4","entry":"conv3x3","repo":"xljh0520/jotr","repo_kind":"official","path":"common/nets/resnet.py","file_url":"https://github.com/xljh0520/jotr/blob/HEAD/common/nets/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"160bb14bd76201b4"}},{"code_sha256_prefix":"c7c51a0c08be8490","entry":"intra_info_nce_loss","repo":"xljh0520/jotr","repo_kind":"official","path":"common/nets/infonce.py","file_url":"https://github.com/xljh0520/jotr/blob/HEAD/common/nets/infonce.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c7c51a0c08be8490"}},{"code_sha256_prefix":"83c03ceb1307c6c4","entry":"inverse_sigmoid","repo":"xljh0520/jotr","repo_kind":"official","path":"common/nets/transformer.py","file_url":"https://github.com/xljh0520/jotr/blob/HEAD/common/nets/transformer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"83c03ceb1307c6c4"}},{"code_sha256_prefix":"41843763717707cc","entry":"make_conv1d_layers","repo":"xljh0520/jotr","repo_kind":"official","path":"common/nets/layer.py","file_url":"https://github.com/xljh0520/jotr/blob/HEAD/common/nets/layer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"41843763717707cc"}},{"code_sha256_prefix":"622140af456ff5d2","entry":"make_conv_layers","repo":"xljh0520/jotr","repo_kind":"official","path":"common/nets/layer.py","file_url":"https://github.com/xljh0520/jotr/blob/HEAD/common/nets/layer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"622140af456ff5d2"}},{"code_sha256_prefix":"6309528f3f76d031","entry":"make_linear_layers","repo":"xljh0520/jotr","repo_kind":"official","path":"common/nets/layer.py","file_url":"https://github.com/xljh0520/jotr/blob/HEAD/common/nets/layer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6309528f3f76d031"}},{"code_sha256_prefix":"59f9b871646fa8fb","entry":"mlp","repo":"xljh0520/jotr","repo_kind":"official","path":"common/model.py","file_url":"https://github.com/xljh0520/jotr/blob/HEAD/common/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59f9b871646fa8fb"}},{"code_sha256_prefix":"55f1419d4d8c483c","entry":"transpose","repo":"xljh0520/jotr","repo_kind":"official","path":"common/nets/info_nce.py","file_url":"https://github.com/xljh0520/jotr/blob/HEAD/common/nets/info_nce.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"55f1419d4d8c483c"}},{"code_sha256_prefix":"6080b5fc53d53bfa","entry":"Transformer","repo":"xljh0520/JOTR","repo_kind":"official","path":"common/nets/transformer.py","file_url":"https://github.com/xljh0520/JOTR/blob/HEAD/common/nets/transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6080b5fc53d53bfa"}},{"code_sha256_prefix":"b4959be409139bd4","entry":"bulid_transformer_decoder","repo":"xljh0520/jotr","repo_kind":"official","path":"common/nets/transformer.py","file_url":"https://github.com/xljh0520/jotr/blob/HEAD/common/nets/transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b4959be409139bd4"}},{"code_sha256_prefix":"54b175c46812e410","entry":"info_nce","repo":"xljh0520/jotr","repo_kind":"official","path":"common/nets/info_nce.py","file_url":"https://github.com/xljh0520/jotr/blob/HEAD/common/nets/info_nce.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"54b175c46812e410"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}