{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mae-dfer-efficient-masked-autoencoder-for","title":"MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognition","arxiv_id":"2307.02227","date":"2023-07-05","proceeding":null,"authors":["Licai Sun","Zheng Lian","Bin Liu","JianHua Tao"],"abstract":"Dynamic facial expression recognition (DFER) is essential to the development of intelligent and empathetic machines. Prior efforts in this field mainly fall into supervised learning paradigm, which is severely restricted by the limited labeled data in existing datasets. Inspired by recent unprecedented success of masked autoencoders (e.g., VideoMAE), this paper proposes MAE-DFER, a novel self-supervised method which leverages large-scale self-supervised pre-training on abundant unlabeled data to largely advance the development of DFER. Since the vanilla Vision Transformer (ViT) employed in VideoMAE requires substantial computation during fine-tuning, MAE-DFER develops an efficient local-global interaction Transformer (LGI-Former) as the encoder. Moreover, in addition to the standalone appearance content reconstruction in VideoMAE, MAE-DFER also introduces explicit temporal facial motion modeling to encourage LGI-Former to excavate both static appearance and dynamic motion information. Extensive experiments on six datasets show that MAE-DFER consistently outperforms state-of-the-art supervised methods by significant margins (e.g., +6.30\\% UAR on DFEW and +8.34\\% UAR on MAFW), verifying that it can learn powerful dynamic facial representations via large-scale self-supervised pre-training. Besides, it has comparable or even better performance than VideoMAE, while largely reducing the computational cost (about 38\\% FLOPs). We believe MAE-DFER has paved a new way for the advancement of DFER and can inspire more relevant research in this field and even other related tasks. Codes and models are publicly available at https://github.com/sunlicai/MAE-DFER.","url_abs":"https://arxiv.org/abs/2307.02227v2","url_pdf":"https://arxiv.org/pdf/2307.02227v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mae-dfer-efficient-masked-autoencoder-for","repo_url":"https://github.com/sunlicai/mae-dfer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"dynamic-facial-expression-recognition","task_name":"Dynamic Facial Expression Recognition"},{"task_slug":"facial-expression-recognition-1","task_name":"Facial Expression Recognition"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2307.02227","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.02227"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sunlicai/mae-dfer","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":7,"ran_honours":1,"ran_draft_wrong":1,"unverified":5},"by_repo_kind":{"official":{"samples":14,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bc39b8ed5fefa7eb","entry":"crop_clip","repo":"sunlicai/mae-dfer","repo_kind":"official","path":"functional.py","file_url":"https://github.com/sunlicai/mae-dfer/blob/HEAD/functional.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc39b8ed5fefa7eb"}},{"code_sha256_prefix":"220573d3a318e31b","entry":"get_num_layer_for_avit","repo":"sunlicai/mae-dfer","repo_kind":"official","path":"optim_factory.py","file_url":"https://github.com/sunlicai/mae-dfer/blob/HEAD/optim_factory.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"220573d3a318e31b"}},{"code_sha256_prefix":"a3448702d6025ce6","entry":"get_num_layer_for_vit","repo":"sunlicai/mae-dfer","repo_kind":"official","path":"optim_factory.py","file_url":"https://github.com/sunlicai/mae-dfer/blob/HEAD/optim_factory.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a3448702d6025ce6"}},{"code_sha256_prefix":"2b53e457d5632b4e","entry":"get_resize_sizes","repo":"sunlicai/mae-dfer","repo_kind":"official","path":"functional.py","file_url":"https://github.com/sunlicai/mae-dfer/blob/HEAD/functional.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b53e457d5632b4e"}},{"code_sha256_prefix":"9b2250a3d13ed688","entry":"get_sinusoid_encoding_table","repo":"sunlicai/mae-dfer","repo_kind":"official","path":"modeling_finetune.py","file_url":"https://github.com/sunlicai/mae-dfer/blob/HEAD/modeling_finetune.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b2250a3d13ed688"}},{"code_sha256_prefix":"8733e0d87f232d11","entry":"mixup_target","repo":"sunlicai/mae-dfer","repo_kind":"official","path":"mixup.py","file_url":"https://github.com/sunlicai/mae-dfer/blob/HEAD/mixup.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8733e0d87f232d11"}},{"code_sha256_prefix":"4728dc2025227bba","entry":"one_hot","repo":"sunlicai/mae-dfer","repo_kind":"official","path":"mixup.py","file_url":"https://github.com/sunlicai/mae-dfer/blob/HEAD/mixup.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4728dc2025227bba"}},{"code_sha256_prefix":"3913c620f6926f42","entry":"rand_bbox","repo":"sunlicai/mae-dfer","repo_kind":"official","path":"mixup.py","file_url":"https://github.com/sunlicai/mae-dfer/blob/HEAD/mixup.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3913c620f6926f42"}},{"code_sha256_prefix":"981936c73d3cdd12","entry":"resize_clip","repo":"sunlicai/mae-dfer","repo_kind":"official","path":"functional.py","file_url":"https://github.com/sunlicai/mae-dfer/blob/HEAD/functional.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"981936c73d3cdd12"}},{"code_sha256_prefix":"29416043c7035e7c","entry":"get_loss_scale_for_deepspeed","repo":"sunlicai/mae-dfer","repo_kind":"official","path":"engine_for_finetuning.py","file_url":"https://github.com/sunlicai/mae-dfer/blob/HEAD/engine_for_finetuning.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"29416043c7035e7c"}},{"code_sha256_prefix":"cbc7a2132a7cd33f","entry":"get_model","repo":"sunlicai/mae-dfer","repo_kind":"official","path":"run_mae_pretraining.py","file_url":"https://github.com/sunlicai/mae-dfer/blob/HEAD/run_mae_pretraining.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cbc7a2132a7cd33f"}},{"code_sha256_prefix":"0ef0019514eaaa09","entry":"get_parameter_groups","repo":"sunlicai/mae-dfer","repo_kind":"official","path":"optim_factory.py","file_url":"https://github.com/sunlicai/mae-dfer/blob/HEAD/optim_factory.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ef0019514eaaa09"}},{"code_sha256_prefix":"14b3b13b2e7ee844","entry":"tensor_normalize","repo":"sunlicai/mae-dfer","repo_kind":"official","path":"kinetics.py","file_url":"https://github.com/sunlicai/mae-dfer/blob/HEAD/kinetics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"14b3b13b2e7ee844"}},{"code_sha256_prefix":"c7976ea27edc377a","entry":"train_class_batch","repo":"sunlicai/mae-dfer","repo_kind":"official","path":"engine_for_finetuning.py","file_url":"https://github.com/sunlicai/mae-dfer/blob/HEAD/engine_for_finetuning.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c7976ea27edc377a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}