{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/in-search-of-a-robust-facial-expressions","title":"In Search of a Robust Facial Expressions Recognition Model: A Large-Scale Visual Cross-Corpus Study","arxiv_id":null,"date":"2022-10-07","proceeding":"Neurocomputing 2022 10","authors":["Elena Ryumina","Denis Dresvyanskiy","Alexey Karpov"],"abstract":"Many researchers have been seeking robust emotion recognition system for already last two decades. It would advance computer systems to a new level of interaction, providing much more natural feedback during human–computer interaction due to analysis of user affect state. However, one of the key problems in this domain is a lack of generalization ability: we observe dramatic degradation of model performance when it was trained on one corpus and evaluated on another one. Although some studies were done in this direction, visual modality still remains under-investigated. Therefore, we introduce the visual cross-corpus study conducted with the utilization of eight corpora, which differ in recording conditions, participants’ appearance characteristics, and complexity of data processing. We propose a visual-based end-to-end emotion recognition framework, which consists of the robust pre-trained backbone model and temporal sub-system in order to model temporal dependencies across many video frames. In addition, a detailed analysis of mistakes and advantages of the backbone model is provided, demonstrating its high ability of generalization. Our results show that the backbone model has achieved the accuracy of 66.4% on the AffectNet dataset, outperforming all the state-of-the-art results. Moreover, the CNN-LSTM model has demonstrated a decent efficacy on dynamic visual datasets during cross-corpus experiments, achieving comparable with state-of-the-art results. In addition, we provide backbone and CNN-LSTM models for future researchers: they can be accessed via GitHub.","url_abs":"https://www.sciencedirect.com/science/article/abs/pii/S0925231222012656","url_pdf":"https://www.sciencedirect.com/science/article/abs/pii/S0925231222012656","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"in-search-of-a-robust-facial-expressions","repo_url":"https://github.com/ElenaRyumina/EMO-AffectNetModel","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"cross-corpus","task_name":"Cross-corpus"},{"task_slug":"emotion-recognition","task_name":"Emotion Recognition"},{"task_slug":"face-detection","task_name":"Face Detection"},{"task_slug":"facial-expression-recognition","task_name":"Facial Expression Recognition (FER)"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"mixup","method_name":"Mixup"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/facial-expression-recognition-on-aff-wild2","task":"Facial Expression Recognition (FER)","dataset":"Aff-Wild2","model":"EmoAffectNet LSTM","rank_in_archive_order":2,"of":2,"metrics":{"UAR":"52.9"},"uses_additional_data":false},{"leaderboard":"/sota/facial-expression-recognition-on-affectnet","task":"Facial Expression Recognition (FER)","dataset":"AffectNet","model":"EmoAffectNet","rank_in_archive_order":39,"of":50,"metrics":{"Accuracy (7 emotion)":"66.49"},"uses_additional_data":true},{"leaderboard":"/sota/facial-expression-recognition-on-crema-d","task":"Facial Expression Recognition (FER)","dataset":"CREMA-D","model":"EmoAffectNet LSTM","rank_in_archive_order":1,"of":1,"metrics":{"UAR":"79.0"},"uses_additional_data":false},{"leaderboard":"/sota/facial-expression-recognition-on-ravdess","task":"Facial Expression Recognition (FER)","dataset":"RAVDESS","model":"EmoAffectNet LSTM","rank_in_archive_order":1,"of":1,"metrics":{"UAR":"69.7"},"uses_additional_data":false},{"leaderboard":"/sota/facial-expression-recognition-on-savee","task":"Facial Expression Recognition (FER)","dataset":"SAVEE","model":"EmoAffectNet LSTM","rank_in_archive_order":1,"of":1,"metrics":{"UAR":"82.8"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}