{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/avatarme-facial-shape-and-brdf-inference-with","title":"AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANs","arxiv_id":"2112.05957","date":"2021-12-11","proceeding":null,"authors":["Alexandros Lattas","Stylianos Moschoglou","Stylianos Ploumpis","Baris Gecer","Abhijeet Ghosh","Stefanos Zafeiriou"],"abstract":"Over the last years, many face analysis tasks have accomplished astounding performance, with applications including face generation and 3D face reconstruction from a single \"in-the-wild\" image. Nevertheless, to the best of our knowledge, there is no method which can produce render-ready high-resolution 3D faces from \"in-the-wild\" images and this can be attributed to the: (a) scarcity of available data for training, and (b) lack of robust methodologies that can successfully be applied on very high-resolution data. In this work, we introduce the first method that is able to reconstruct photorealistic render-ready 3D facial geometry and BRDF from a single \"in-the-wild\" image. We capture a large dataset of facial shape and reflectance, which we have made public. We define a fast facial photorealistic differentiable rendering methodology with accurate facial skin diffuse and specular reflection, self-occlusion and subsurface scattering approximation. With this, we train a network that disentangles the facial diffuse and specular BRDF components from a shape and texture with baked illumination, reconstructed with a state-of-the-art 3DMM fitting method. Our method outperforms the existing arts by a significant margin and reconstructs high-resolution 3D faces from a single low-resolution image, that can be rendered in various applications, and bridge the uncanny valley.","url_abs":"https://arxiv.org/abs/2112.05957v1","url_pdf":"https://arxiv.org/pdf/2112.05957v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"avatarme-facial-shape-and-brdf-inference-with","repo_url":"https://github.com/lattas/avatarme","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"3d-face-reconstruction","task_name":"3D Face Reconstruction"},{"task_slug":"face-generation","task_name":"Face Generation"},{"task_slug":"face-reconstruction","task_name":"Face Reconstruction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2112.05957","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}