{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ganfit-generative-adversarial-network-fitting","title":"GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction","arxiv_id":"1902.05978","date":"2019-02-15","proceeding":"CVPR 2019 6","authors":["Baris Gecer","Stylianos Ploumpis","Irene Kotsia","Stefanos Zafeiriou"],"abstract":"In the past few years, a lot of work has been done towards reconstructing the\n3D facial structure from single images by capitalizing on the power of Deep\nConvolutional Neural Networks (DCNNs). In the most recent works, differentiable\nrenderers were employed in order to learn the relationship between the facial\nidentity features and the parameters of a 3D morphable model for shape and\ntexture. The texture features either correspond to components of a linear\ntexture space or are learned by auto-encoders directly from in-the-wild images.\nIn all cases, the quality of the facial texture reconstruction of the\nstate-of-the-art methods is still not capable of modeling textures in high\nfidelity. In this paper, we take a radically different approach and harness the\npower of Generative Adversarial Networks (GANs) and DCNNs in order to\nreconstruct the facial texture and shape from single images. That is, we\nutilize GANs to train a very powerful generator of facial texture in UV space.\nThen, we revisit the original 3D Morphable Models (3DMMs) fitting approaches\nmaking use of non-linear optimization to find the optimal latent parameters\nthat best reconstruct the test image but under a new perspective. We optimize\nthe parameters with the supervision of pretrained deep identity features\nthrough our end-to-end differentiable framework. We demonstrate excellent\nresults in photorealistic and identity preserving 3D face reconstructions and\nachieve for the first time, to the best of our knowledge, facial texture\nreconstruction with high-frequency details.","url_abs":"http://arxiv.org/abs/1902.05978v2","url_pdf":"http://arxiv.org/pdf/1902.05978v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ganfit-generative-adversarial-network-fitting","repo_url":"https://github.com/barisgecer/ganfit","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[{"task_slug":"3d-face-reconstruction","task_name":"3D Face Reconstruction"},{"task_slug":"face-reconstruction","task_name":"Face Reconstruction"},{"task_slug":null,"task_name":"Generative Adversarial Network"},{"task_slug":"high","task_name":"Vocal Bursts Intensity Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-face-reconstruction-on-florence","task":"3D Face Reconstruction","dataset":"Florence","model":"GANFit","rank_in_archive_order":4,"of":16,"metrics":{"Average 3D Error":"0.95"},"uses_additional_data":false},{"leaderboard":"/sota/3d-face-reconstruction-on-realy","task":"3D Face Reconstruction","dataset":"REALY","model":"GANFit","rank_in_archive_order":10,"of":24,"metrics":{"@cheek":"1.329 (±0.504)","@forehead":"2.402 (±0.545)","@mouth":"1.812 (±0.544)","@nose":"1.928 (±0.490)","all":"1.868"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1902.05978","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}