{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/latent-ofer-detect-mask-and-reconstruct-with","title":"Latent-OFER: Detect, Mask, and Reconstruct with Latent Vectors for Occluded Facial Expression Recognition","arxiv_id":"2307.11404","date":"2023-07-21","proceeding":"ICCV 2023 1","authors":["Isack Lee","Eungi Lee","Seok Bong Yoo"],"abstract":"Most research on facial expression recognition (FER) is conducted in highly controlled environments, but its performance is often unacceptable when applied to real-world situations. This is because when unexpected objects occlude the face, the FER network faces difficulties extracting facial features and accurately predicting facial expressions. Therefore, occluded FER (OFER) is a challenging problem. Previous studies on occlusion-aware FER have typically required fully annotated facial images for training. However, collecting facial images with various occlusions and expression annotations is time-consuming and expensive. Latent-OFER, the proposed method, can detect occlusions, restore occluded parts of the face as if they were unoccluded, and recognize them, improving FER accuracy. This approach involves three steps: First, the vision transformer (ViT)-based occlusion patch detector masks the occluded position by training only latent vectors from the unoccluded patches using the support vector data description algorithm. Second, the hybrid reconstruction network generates the masking position as a complete image using the ViT and convolutional neural network (CNN). Last, the expression-relevant latent vector extractor retrieves and uses expression-related information from all latent vectors by applying a CNN-based class activation map. This mechanism has a significant advantage in preventing performance degradation from occlusion by unseen objects. The experimental results on several databases demonstrate the superiority of the proposed method over state-of-the-art methods.","url_abs":"https://arxiv.org/abs/2307.11404v1","url_pdf":"https://arxiv.org/pdf/2307.11404v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"latent-ofer-detect-mask-and-reconstruct-with","repo_url":"https://github.com/leeisack/latent-ofer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"facial-expression-recognition-1","task_name":"Facial Expression Recognition"},{"task_slug":"facial-expression-recognition","task_name":"Facial Expression Recognition (FER)"},{"task_slug":null,"task_name":"Position"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2307.11404","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.11404"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/leeisack/latent-ofer","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/leeisack/Latent-OFER","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":7,"unverified":1},"by_repo_kind":{"official":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"16c91a259766f1b1","entry":"ChannelAttention","repo":"leeisack/Latent-OFER","repo_kind":"official","path":"networks/OURS.py","file_url":"https://github.com/leeisack/Latent-OFER/blob/HEAD/networks/OURS.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"16c91a259766f1b1"}},{"code_sha256_prefix":"9074aaa8c12e5d97","entry":"CrossAttentionHead","repo":"leeisack/Latent-OFER","repo_kind":"official","path":"networks/OURS.py","file_url":"https://github.com/leeisack/Latent-OFER/blob/HEAD/networks/OURS.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9074aaa8c12e5d97"}},{"code_sha256_prefix":"22d6ff59ee479e9e","entry":"OURS","repo":"leeisack/Latent-OFER","repo_kind":"official","path":"networks/OURS.py","file_url":"https://github.com/leeisack/Latent-OFER/blob/HEAD/networks/OURS.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"22d6ff59ee479e9e"}},{"code_sha256_prefix":"a1215e966402b806","entry":"SpatialAttention","repo":"leeisack/Latent-OFER","repo_kind":"official","path":"networks/OURS.py","file_url":"https://github.com/leeisack/Latent-OFER/blob/HEAD/networks/OURS.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a1215e966402b806"}},{"code_sha256_prefix":"f8060329b57192bd","entry":"get_norm_layer","repo":"leeisack/latent-ofer","repo_kind":"official","path":"hybrid_recon/models/networks.py","file_url":"https://github.com/leeisack/latent-ofer/blob/HEAD/hybrid_recon/models/networks.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f8060329b57192bd"}},{"code_sha256_prefix":"d2575c24d11029c0","entry":"get_scheduler","repo":"leeisack/latent-ofer","repo_kind":"official","path":"hybrid_recon/models/networks.py","file_url":"https://github.com/leeisack/latent-ofer/blob/HEAD/hybrid_recon/models/networks.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d2575c24d11029c0"}},{"code_sha256_prefix":"5a2166bc64a34f5c","entry":"init_net","repo":"leeisack/latent-ofer","repo_kind":"official","path":"hybrid_recon/models/networks.py","file_url":"https://github.com/leeisack/latent-ofer/blob/HEAD/hybrid_recon/models/networks.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5a2166bc64a34f5c"}},{"code_sha256_prefix":"51fdefc0e1bd9dc6","entry":"create_model","repo":"leeisack/latent-ofer","repo_kind":"official","path":"hybrid_recon/models/models.py","file_url":"https://github.com/leeisack/latent-ofer/blob/HEAD/hybrid_recon/models/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"51fdefc0e1bd9dc6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}