{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fine-grained-multi-view-hand-reconstruction","title":"Fine-Grained Multi-View Hand Reconstruction Using Inverse Rendering","arxiv_id":"2407.05680","date":"2024-07-08","proceeding":null,"authors":["Qijun Gan","Wentong Li","Jinwei Ren","Jianke Zhu"],"abstract":"Reconstructing high-fidelity hand models with intricate textures plays a crucial role in enhancing human-object interaction and advancing real-world applications. Despite the state-of-the-art methods excelling in texture generation and image rendering, they often face challenges in accurately capturing geometric details. Learning-based approaches usually offer better robustness and faster inference, which tend to produce smoother results and require substantial amounts of training data. To address these issues, we present a novel fine-grained multi-view hand mesh reconstruction method that leverages inverse rendering to restore hand poses and intricate details. Firstly, our approach predicts a parametric hand mesh model through Graph Convolutional Networks (GCN) based method from multi-view images. We further introduce a novel Hand Albedo and Mesh (HAM) optimization module to refine both the hand mesh and textures, which is capable of preserving the mesh topology. In addition, we suggest an effective mesh-based neural rendering scheme to simultaneously generate photo-realistic image and optimize mesh geometry by fusing the pre-trained rendering network with vertex features. We conduct the comprehensive experiments on InterHand2.6M, DeepHandMesh and dataset collected by ourself, whose promising results show that our proposed approach outperforms the state-of-the-art methods on both reconstruction accuracy and rendering quality. Code and dataset are publicly available at https://github.com/agnJason/FMHR.","url_abs":"https://arxiv.org/abs/2407.05680v2","url_pdf":"https://arxiv.org/pdf/2407.05680v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fine-grained-multi-view-hand-reconstruction","repo_url":"https://github.com/agnjason/fmhr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"human-object-interaction-detection","task_name":"Human-Object Interaction Detection"},{"task_slug":"inverse-rendering","task_name":"Inverse Rendering"},{"task_slug":"neural-rendering","task_name":"Neural Rendering"},{"task_slug":"texture-synthesis","task_name":"Texture Synthesis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.05680","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.05680"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/agnjason/fmhr","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/agnJason/FMHR","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":22,"ran_fixture":3},"by_repo_kind":{"official":{"samples":25,"ran":25,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"28a53e0f276bbbc8","entry":"NCC","repo":"agnjason/fmhr","repo_kind":"official","path":"models/ncc_utils.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/ncc_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"28a53e0f276bbbc8"}},{"code_sha256_prefix":"c27682ba51eb4e6f","entry":"add_hom","repo":"agnjason/fmhr","repo_kind":"official","path":"models/ncc_utils.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/ncc_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c27682ba51eb4e6f"}},{"code_sha256_prefix":"e7c54f2a25d9ee14","entry":"calculate_psnr","repo":"agnjason/fmhr","repo_kind":"official","path":"train_unet.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/train_unet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e7c54f2a25d9ee14"}},{"code_sha256_prefix":"1725abb601f10ca5","entry":"compute_laplacian","repo":"agnjason/fmhr","repo_kind":"official","path":"mano_optim.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/mano_optim.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1725abb601f10ca5"}},{"code_sha256_prefix":"999ef3f8561e7b10","entry":"convert2sparse","repo":"agnjason/fmhr","repo_kind":"official","path":"models/gcn.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/gcn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"999ef3f8561e7b10"}},{"code_sha256_prefix":"9583f37b904687d4","entry":"convmtx2","repo":"agnjason/fmhr","repo_kind":"official","path":"models/sfs_utils.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/sfs_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9583f37b904687d4"}},{"code_sha256_prefix":"08ca1a3c5b8af575","entry":"depth2normal","repo":"agnjason/fmhr","repo_kind":"official","path":"models/sfs_utils.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/sfs_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"08ca1a3c5b8af575"}},{"code_sha256_prefix":"3359d23ac8fc9fd1","entry":"estimate_shading","repo":"agnjason/fmhr","repo_kind":"official","path":"models/sfs_utils.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/sfs_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3359d23ac8fc9fd1"}},{"code_sha256_prefix":"086ad4999b6c210e","entry":"fftfreqs","repo":"agnjason/fmhr","repo_kind":"official","path":"models/sap.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/sap.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"086ad4999b6c210e"}},{"code_sha256_prefix":"a84f1093efd108a8","entry":"get_edges","repo":"agnjason/fmhr","repo_kind":"official","path":"mano_optim.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/mano_optim.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a84f1093efd108a8"}},{"code_sha256_prefix":"e19803e9ee2c4627","entry":"get_ray_directions","repo":"agnjason/fmhr","repo_kind":"official","path":"models/get_rays.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/get_rays.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e19803e9ee2c4627"}},{"code_sha256_prefix":"f82ba403be143b44","entry":"get_rays","repo":"agnjason/fmhr","repo_kind":"official","path":"models/get_rays.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/get_rays.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f82ba403be143b44"}},{"code_sha256_prefix":"d77c3b9fe4c2a271","entry":"img","repo":"agnjason/fmhr","repo_kind":"official","path":"models/sap.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/sap.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d77c3b9fe4c2a271"}},{"code_sha256_prefix":"10908dad0c9e096d","entry":"img_contrast_bright","repo":"agnjason/fmhr","repo_kind":"official","path":"get_data.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/get_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"10908dad0c9e096d"}},{"code_sha256_prefix":"278d8710708c571f","entry":"laplacian","repo":"agnjason/fmhr","repo_kind":"official","path":"models/mesh_sampling.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/mesh_sampling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"278d8710708c571f"}},{"code_sha256_prefix":"635669225b79bf5c","entry":"laplacian_cot","repo":"agnjason/fmhr","repo_kind":"official","path":"mano_optim.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/mano_optim.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"635669225b79bf5c"}},{"code_sha256_prefix":"ee38789614e9343f","entry":"normalize","repo":"agnjason/fmhr","repo_kind":"official","path":"models/ncc_utils.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/ncc_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ee38789614e9343f"}},{"code_sha256_prefix":"3adcb35add712974","entry":"poolwT","repo":"agnjason/fmhr","repo_kind":"official","path":"models/gcn.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/gcn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3adcb35add712974"}},{"code_sha256_prefix":"e331855d53ef4b5e","entry":"rescale_L","repo":"agnjason/fmhr","repo_kind":"official","path":"models/mesh_sampling.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/mesh_sampling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e331855d53ef4b5e"}},{"code_sha256_prefix":"523ed667113feb04","entry":"sap_generate","repo":"agnJason/FMHR","repo_kind":"official","path":"models/sap.py","file_url":"https://github.com/agnJason/FMHR/blob/HEAD/models/sap.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"523ed667113feb04"}},{"code_sha256_prefix":"f3a258f73e2c917f","entry":"sap_transform","repo":"agnJason/FMHR","repo_kind":"official","path":"models/sap.py","file_url":"https://github.com/agnJason/FMHR/blob/HEAD/models/sap.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f3a258f73e2c917f"}},{"code_sha256_prefix":"ff4a0c727a4528a5","entry":"scale_input","repo":"agnjason/fmhr","repo_kind":"official","path":"models/PostionalEncoding.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/PostionalEncoding.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ff4a0c727a4528a5"}},{"code_sha256_prefix":"656da6fdcc2cdcbf","entry":"spec_gaussian_filter","repo":"agnjason/fmhr","repo_kind":"official","path":"models/sap.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/sap.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"656da6fdcc2cdcbf"}},{"code_sha256_prefix":"1773bfa986183e62","entry":"transform_3D_grid","repo":"agnjason/fmhr","repo_kind":"official","path":"models/PostionalEncoding.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/PostionalEncoding.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1773bfa986183e62"}},{"code_sha256_prefix":"3129a88f8d186902","entry":"vertex_quadrics","repo":"agnjason/fmhr","repo_kind":"official","path":"models/mesh_sampling.py","file_url":"https://github.com/agnjason/fmhr/blob/HEAD/models/mesh_sampling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3129a88f8d186902"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}