{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/3d-morphable-models-as-spatial-transformer","title":"3D Morphable Models as Spatial Transformer Networks","arxiv_id":"1708.07199","date":"2017-08-23","proceeding":null,"authors":["Anil Bas","Patrik Huber","William A. P. Smith","Muhammad Awais","Josef Kittler"],"abstract":"In this paper, we show how a 3D Morphable Model (i.e. a statistical model of\nthe 3D shape of a class of objects such as faces) can be used to spatially\ntransform input data as a module (a 3DMM-STN) within a convolutional neural\nnetwork. This is an extension of the original spatial transformer network in\nthat we are able to interpret and normalise 3D pose changes and\nself-occlusions. The trained localisation part of the network is independently\nuseful since it learns to fit a 3D morphable model to a single image. We show\nthat the localiser can be trained using only simple geometric loss functions on\na relatively small dataset yet is able to perform robust normalisation on\nhighly uncontrolled images including occlusion, self-occlusion and large pose\nchanges.","url_abs":"http://arxiv.org/abs/1708.07199v1","url_pdf":"http://arxiv.org/pdf/1708.07199v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"3d-morphable-models-as-spatial-transformer","repo_url":"https://github.com/anilbas/3DMMasSTN","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"spatial-transformer","method_name":"Spatial Transformer"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1708.07199","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}