{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generating-3d-faces-using-convolutional-mesh","title":"Generating 3D faces using Convolutional Mesh Autoencoders","arxiv_id":"1807.10267","date":"2018-07-26","proceeding":"ECCV 2018 9","authors":["Anurag Ranjan","Timo Bolkart","Soubhik Sanyal","Michael J. Black"],"abstract":"Learned 3D representations of human faces are useful for computer vision\nproblems such as 3D face tracking and reconstruction from images, as well as\ngraphics applications such as character generation and animation. Traditional\nmodels learn a latent representation of a face using linear subspaces or\nhigher-order tensor generalizations. Due to this linearity, they can not\ncapture extreme deformations and non-linear expressions. To address this, we\nintroduce a versatile model that learns a non-linear representation of a face\nusing spectral convolutions on a mesh surface. We introduce mesh sampling\noperations that enable a hierarchical mesh representation that captures\nnon-linear variations in shape and expression at multiple scales within the\nmodel. In a variational setting, our model samples diverse realistic 3D faces\nfrom a multivariate Gaussian distribution. Our training data consists of 20,466\nmeshes of extreme expressions captured over 12 different subjects. Despite\nlimited training data, our trained model outperforms state-of-the-art face\nmodels with 50% lower reconstruction error, while using 75% fewer parameters.\nWe also show that, replacing the expression space of an existing\nstate-of-the-art face model with our autoencoder, achieves a lower\nreconstruction error. Our data, model and code are available at\nhttp://github.com/anuragranj/coma","url_abs":"http://arxiv.org/abs/1807.10267v3","url_pdf":"http://arxiv.org/pdf/1807.10267v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generating-3d-faces-using-convolutional-mesh","repo_url":"https://github.com/anuragranj/coma","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"generating-3d-faces-using-convolutional-mesh","repo_url":"https://github.com/QianliM/CAPE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"3d-face-modeling","task_name":"3D Face Modelling"},{"task_slug":"face-alignment","task_name":"Face Alignment"},{"task_slug":"face-model","task_name":"Face Model"}],"methods":[],"datasets_introduced":[{"slug":"coma","name":"COMA","full_name":"COMA"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/face-alignment-on-facescape","task":"Face Alignment","dataset":"FaceScape","model":"CoMA","rank_in_archive_order":4,"of":4,"metrics":{"NME":"1.088"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1807.10267","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}