{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/neural-face-rigging-for-animating-and","title":"Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild","arxiv_id":"2305.08296","date":"2023-05-15","proceeding":null,"authors":["Dafei Qin","Jun Saito","Noam Aigerman","Thibault Groueix","Taku Komura"],"abstract":"We propose an end-to-end deep-learning approach for automatic rigging and retargeting of 3D models of human faces in the wild. Our approach, called Neural Face Rigging (NFR), holds three key properties: (i) NFR's expression space maintains human-interpretable editing parameters for artistic controls; (ii) NFR is readily applicable to arbitrary facial meshes with different connectivity and expressions; (iii) NFR can encode and produce fine-grained details of complex expressions performed by arbitrary subjects. To the best of our knowledge, NFR is the first approach to provide realistic and controllable deformations of in-the-wild facial meshes, without the manual creation of blendshapes or correspondence. We design a deformation autoencoder and train it through a multi-dataset training scheme, which benefits from the unique advantages of two data sources: a linear 3DMM with interpretable control parameters as in FACS, and 4D captures of real faces with fine-grained details. Through various experiments, we show NFR's ability to automatically produce realistic and accurate facial deformations across a wide range of existing datasets as well as noisy facial scans in-the-wild, while providing artist-controlled, editable parameters.","url_abs":"https://arxiv.org/abs/2305.08296v1","url_pdf":"https://arxiv.org/pdf/2305.08296v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"neural-face-rigging-for-animating-and","repo_url":"https://github.com/dafei-qin/NFR_pytorch","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"nfr","method_name":"NFR"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.08296","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.08296"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dafei-qin/NFR_pytorch","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":9},"by_repo_kind":{"official":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ba53eec4cd8756e5","entry":"Loss","repo":"dafei-qin/NFR_pytorch","repo_kind":"official","path":"dataset_preprocess.py","file_url":"https://github.com/dafei-qin/NFR_pytorch/blob/HEAD/dataset_preprocess.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ba53eec4cd8756e5"}},{"code_sha256_prefix":"df2d65dfef54b47d","entry":"calc_loss","repo":"dafei-qin/NFR_pytorch","repo_kind":"official","path":"FWH_run.py","file_url":"https://github.com/dafei-qin/NFR_pytorch/blob/HEAD/FWH_run.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df2d65dfef54b47d"}},{"code_sha256_prefix":"8bfc34ee66246512","entry":"calc_qr","repo":"dafei-qin/NFR_pytorch","repo_kind":"official","path":"deformation_transfer.py","file_url":"https://github.com/dafei-qin/NFR_pytorch/blob/HEAD/deformation_transfer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8bfc34ee66246512"}},{"code_sha256_prefix":"e44ef825ac05f42f","entry":"calc_transform","repo":"dafei-qin/NFR_pytorch","repo_kind":"official","path":"deformation_transfer.py","file_url":"https://github.com/dafei-qin/NFR_pytorch/blob/HEAD/deformation_transfer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e44ef825ac05f42f"}},{"code_sha256_prefix":"f67ecdf49a2a7214","entry":"feature_transform_reguliarzer","repo":"dafei-qin/NFR_pytorch","repo_kind":"official","path":"mymodel.py","file_url":"https://github.com/dafei-qin/NFR_pytorch/blob/HEAD/mymodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f67ecdf49a2a7214"}},{"code_sha256_prefix":"06167d80ef5b11ae","entry":"get_centriod","repo":"dafei-qin/NFR_pytorch","repo_kind":"official","path":"dataset_preprocess.py","file_url":"https://github.com/dafei-qin/NFR_pytorch/blob/HEAD/dataset_preprocess.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"06167d80ef5b11ae"}},{"code_sha256_prefix":"34ac4d88b7af24be","entry":"get_centroid","repo":"dafei-qin/NFR_pytorch","repo_kind":"official","path":"datasets.py","file_url":"https://github.com/dafei-qin/NFR_pytorch/blob/HEAD/datasets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"34ac4d88b7af24be"}},{"code_sha256_prefix":"721f052ddc43e2c2","entry":"get_normal","repo":"dafei-qin/NFR_pytorch","repo_kind":"official","path":"dataset_preprocess.py","file_url":"https://github.com/dafei-qin/NFR_pytorch/blob/HEAD/dataset_preprocess.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"721f052ddc43e2c2"}},{"code_sha256_prefix":"9e05244dd7c208c7","entry":"get_normal","repo":"dafei-qin/NFR_pytorch","repo_kind":"official","path":"datasets.py","file_url":"https://github.com/dafei-qin/NFR_pytorch/blob/HEAD/datasets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9e05244dd7c208c7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}