{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/facial-expression-recognition-using-vanilla","title":"Emotion Separation and Recognition from a Facial Expression by Generating the Poker Face with Vision Transformers","arxiv_id":"2207.11081","date":"2022-07-22","proceeding":null,"authors":["Jia Li","Jiantao Nie","Dan Guo","Richang Hong","Meng Wang"],"abstract":"Representation learning and feature disentanglement have garnered significant research interest in the field of facial expression recognition (FER). The inherent ambiguity of emotion labels poses challenges for conventional supervised representation learning methods. Moreover, directly learning the mapping from a facial expression image to an emotion label lacks explicit supervision signals for capturing fine-grained facial features. In this paper, we propose a novel FER model, named Poker Face Vision Transformer or PF-ViT, to address these challenges. PF-ViT aims to separate and recognize the disturbance-agnostic emotion from a static facial image via generating its corresponding poker face, without the need for paired images. Inspired by the Facial Action Coding System, we regard an expressive face as the combined result of a set of facial muscle movements on one's poker face (i.e., an emotionless face). PF-ViT utilizes vanilla Vision Transformers, and its components are firstly pre-trained as Masked Autoencoders on a large facial expression dataset without emotion labels, yielding excellent representations. Subsequently, we train PF-ViT using a GAN framework. During training, the auxiliary task of poke face generation promotes the disentanglement between emotional and emotion-irrelevant components, guiding the FER model to holistically capture discriminative facial details. Quantitative and qualitative results demonstrate the effectiveness of our method, surpassing the state-of-the-art methods on four popular FER datasets.","url_abs":"https://arxiv.org/abs/2207.11081v4","url_pdf":"https://arxiv.org/pdf/2207.11081v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"disentanglement","task_name":"Disentanglement"},{"task_slug":"face-generation","task_name":"Face Generation"},{"task_slug":"facial-expression-recognition-1","task_name":"Facial Expression Recognition"},{"task_slug":"facial-expression-recognition","task_name":"Facial Expression Recognition (FER)"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"mae","method_name":"MAE"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/facial-expression-recognition-on-affectnet","task":"Facial Expression Recognition (FER)","dataset":"AffectNet","model":"Vit-base + MAE","rank_in_archive_order":14,"of":50,"metrics":{"Accuracy (8 emotion)":"62.42"},"uses_additional_data":false},{"leaderboard":"/sota/facial-expression-recognition-on-affectnet","task":"Facial Expression Recognition (FER)","dataset":"AffectNet","model":"ViT-tiny","rank_in_archive_order":31,"of":50,"metrics":{"Accuracy (8 emotion)":"58.28"},"uses_additional_data":false},{"leaderboard":"/sota/facial-expression-recognition-on-affectnet","task":"Facial Expression Recognition (FER)","dataset":"AffectNet","model":"ViT-base","rank_in_archive_order":33,"of":50,"metrics":{"Accuracy (8 emotion)":"57.99"},"uses_additional_data":false},{"leaderboard":"/sota/facial-expression-recognition-on-fer-1","task":"Facial Expression Recognition (FER)","dataset":"FER+","model":"Vit-base + MAE","rank_in_archive_order":7,"of":14,"metrics":{"Accuracy":"90.18"},"uses_additional_data":false},{"leaderboard":"/sota/facial-expression-recognition-on-fer-1","task":"Facial Expression Recognition (FER)","dataset":"FER+","model":"ViT-base","rank_in_archive_order":11,"of":14,"metrics":{"Accuracy":"88.91"},"uses_additional_data":false},{"leaderboard":"/sota/facial-expression-recognition-on-fer-1","task":"Facial Expression Recognition (FER)","dataset":"FER+","model":"ViT-tiny","rank_in_archive_order":12,"of":14,"metrics":{"Accuracy":"88.56"},"uses_additional_data":false},{"leaderboard":"/sota/facial-expression-recognition-on-raf-db","task":"Facial Expression Recognition (FER)","dataset":"RAF-DB","model":"ViT-base + MAE","rank_in_archive_order":13,"of":35,"metrics":{"Overall Accuracy":"91.07"},"uses_additional_data":false},{"leaderboard":"/sota/facial-expression-recognition-on-raf-db","task":"Facial Expression Recognition (FER)","dataset":"RAF-DB","model":"ViT-base","rank_in_archive_order":26,"of":35,"metrics":{"Overall Accuracy":"87.22"},"uses_additional_data":false},{"leaderboard":"/sota/facial-expression-recognition-on-raf-db","task":"Facial Expression Recognition (FER)","dataset":"RAF-DB","model":"ViT-tiny","rank_in_archive_order":27,"of":35,"metrics":{"Overall Accuracy":"87.03"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2207.11081","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}