{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/diffusionrig-learning-personalized-priors-for","title":"DiffusionRig: Learning Personalized Priors for Facial Appearance Editing","arxiv_id":"2304.06711","date":"2023-04-13","proceeding":"CVPR 2023 1","authors":["Zheng Ding","Xuaner Zhang","Zhihao Xia","Lars Jebe","Zhuowen Tu","Xiuming Zhang"],"abstract":"We address the problem of learning person-specific facial priors from a small number (e.g., 20) of portrait photos of the same person. This enables us to edit this specific person's facial appearance, such as expression and lighting, while preserving their identity and high-frequency facial details. Key to our approach, which we dub DiffusionRig, is a diffusion model conditioned on, or \"rigged by,\" crude 3D face models estimated from single in-the-wild images by an off-the-shelf estimator. On a high level, DiffusionRig learns to map simplistic renderings of 3D face models to realistic photos of a given person. Specifically, DiffusionRig is trained in two stages: It first learns generic facial priors from a large-scale face dataset and then person-specific priors from a small portrait photo collection of the person of interest. By learning the CGI-to-photo mapping with such personalized priors, DiffusionRig can \"rig\" the lighting, facial expression, head pose, etc. of a portrait photo, conditioned only on coarse 3D models while preserving this person's identity and other high-frequency characteristics. Qualitative and quantitative experiments show that DiffusionRig outperforms existing approaches in both identity preservation and photorealism. Please see the project website: https://diffusionrig.github.io for the supplemental material, video, code, and data.","url_abs":"https://arxiv.org/abs/2304.06711v1","url_pdf":"https://arxiv.org/pdf/2304.06711v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"diffusionrig-learning-personalized-priors-for","repo_url":"https://github.com/adobe-research/diffusion-rig","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2304.06711","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.06711"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/adobe-research/diffusion-rig","reach":null}],"summary":{"unverified":7},"by_repo_kind":{"official":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"22ef91aa500af651","entry":"AttentionBlock","repo":"adobe-research/diffusion-rig","repo_kind":"official","path":"diffusion/model.py","file_url":"https://github.com/adobe-research/diffusion-rig/blob/HEAD/diffusion/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"22ef91aa500af651"}},{"code_sha256_prefix":"04ee3c96077eb137","entry":"DiffusionRig","repo":"adobe-research/diffusion-rig","repo_kind":"official","path":"diffusion/model.py","file_url":"https://github.com/adobe-research/diffusion-rig/blob/HEAD/diffusion/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"04ee3c96077eb137"}},{"code_sha256_prefix":"a74a21eb500c2979","entry":"ResBlock","repo":"adobe-research/diffusion-rig","repo_kind":"official","path":"diffusion/model.py","file_url":"https://github.com/adobe-research/diffusion-rig/blob/HEAD/diffusion/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a74a21eb500c2979"}},{"code_sha256_prefix":"482a63d7f1598ed8","entry":"TimestepBlock","repo":"adobe-research/diffusion-rig","repo_kind":"official","path":"diffusion/model.py","file_url":"https://github.com/adobe-research/diffusion-rig/blob/HEAD/diffusion/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"482a63d7f1598ed8"}},{"code_sha256_prefix":"fedcf75fd2b2ca46","entry":"TimestepEmbedSequential","repo":"adobe-research/diffusion-rig","repo_kind":"official","path":"diffusion/model.py","file_url":"https://github.com/adobe-research/diffusion-rig/blob/HEAD/diffusion/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"fedcf75fd2b2ca46"}},{"code_sha256_prefix":"62664a62d891810b","entry":"UNetModel","repo":"adobe-research/diffusion-rig","repo_kind":"official","path":"diffusion/model.py","file_url":"https://github.com/adobe-research/diffusion-rig/blob/HEAD/diffusion/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"62664a62d891810b"}},{"code_sha256_prefix":"fb8a7a13dcb8c76d","entry":"create_model","repo":"adobe-research/diffusion-rig","repo_kind":"official","path":"diffusion/model.py","file_url":"https://github.com/adobe-research/diffusion-rig/blob/HEAD/diffusion/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"fb8a7a13dcb8c76d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}