{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/emoportraits-emotion-enhanced-multimodal-one","title":"EMOPortraits: Emotion-enhanced Multimodal One-shot Head Avatars","arxiv_id":"2404.19110","date":"2024-04-29","proceeding":"CVPR 2024 1","authors":["Nikita Drobyshev","Antoni Bigata Casademunt","Konstantinos Vougioukas","Zoe Landgraf","Stavros Petridis","Maja Pantic"],"abstract":"Head avatars animated by visual signals have gained popularity, particularly in cross-driving synthesis where the driver differs from the animated character, a challenging but highly practical approach. The recently presented MegaPortraits model has demonstrated state-of-the-art results in this domain. We conduct a deep examination and evaluation of this model, with a particular focus on its latent space for facial expression descriptors, and uncover several limitations with its ability to express intense face motions. To address these limitations, we propose substantial changes in both training pipeline and model architecture, to introduce our EMOPortraits model, where we: Enhance the model's capability to faithfully support intense, asymmetric face expressions, setting a new state-of-the-art result in the emotion transfer task, surpassing previous methods in both metrics and quality. Incorporate speech-driven mode to our model, achieving top-tier performance in audio-driven facial animation, making it possible to drive source identity through diverse modalities, including visual signal, audio, or a blend of both. We propose a novel multi-view video dataset featuring a wide range of intense and asymmetric facial expressions, filling the gap with absence of such data in existing datasets.","url_abs":"https://arxiv.org/abs/2404.19110v1","url_pdf":"https://arxiv.org/pdf/2404.19110v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"emoportraits-emotion-enhanced-multimodal-one","repo_url":"https://github.com/neeek2303/EMOPortraits","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2404.19110","atlas_url":"https://app.syntology.ai/?focus=2404.19110","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.19110"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/neeek2303/EMOPortraits","reach":null}],"summary":{"ran":4,"ran_draft_wrong":1,"ran_fixture":1,"unverified":1},"by_repo_kind":{"listed":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f8a173f407f1f5db","entry":"ImportanceRenderer","repo":"neeek2303/EMOPortraits","repo_kind":"listed","path":"networks/volumetric_avatar/volume_renderer.py","file_url":"https://github.com/neeek2303/EMOPortraits/blob/HEAD/networks/volumetric_avatar/volume_renderer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f8a173f407f1f5db"}},{"code_sha256_prefix":"80b514efb4380504","entry":"MipRayMarcher2","repo":"neeek2303/EMOPortraits","repo_kind":"listed","path":"networks/volumetric_avatar/volume_renderer.py","file_url":"https://github.com/neeek2303/EMOPortraits/blob/HEAD/networks/volumetric_avatar/volume_renderer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"80b514efb4380504"}},{"code_sha256_prefix":"9fd052e5ad5901b0","entry":"OSGDecoder","repo":"neeek2303/EMOPortraits","repo_kind":"listed","path":"networks/volumetric_avatar/volume_renderer.py","file_url":"https://github.com/neeek2303/EMOPortraits/blob/HEAD/networks/volumetric_avatar/volume_renderer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9fd052e5ad5901b0"}},{"code_sha256_prefix":"b37c10c9de2f7e40","entry":"generate_planes","repo":"neeek2303/EMOPortraits","repo_kind":"listed","path":"networks/volumetric_avatar/volume_renderer.py","file_url":"https://github.com/neeek2303/EMOPortraits/blob/HEAD/networks/volumetric_avatar/volume_renderer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b37c10c9de2f7e40"}},{"code_sha256_prefix":"a9f157f50bfdc4d5","entry":"get_embedder","repo":"neeek2303/EMOPortraits","repo_kind":"listed","path":"networks/volumetric_avatar/volume_renderer.py","file_url":"https://github.com/neeek2303/EMOPortraits/blob/HEAD/networks/volumetric_avatar/volume_renderer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a9f157f50bfdc4d5"}},{"code_sha256_prefix":"8b88a2d7c9dc6ce7","entry":"sample_from_features_all","repo":"neeek2303/EMOPortraits","repo_kind":"listed","path":"networks/volumetric_avatar/volume_renderer.py","file_url":"https://github.com/neeek2303/EMOPortraits/blob/HEAD/networks/volumetric_avatar/volume_renderer.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8b88a2d7c9dc6ce7"}},{"code_sha256_prefix":"8bb13d64dd159959","entry":"VolumeRenderer","repo":"neeek2303/EMOPortraits","repo_kind":"listed","path":"networks/volumetric_avatar/volume_renderer.py","file_url":"https://github.com/neeek2303/EMOPortraits/blob/HEAD/networks/volumetric_avatar/volume_renderer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8bb13d64dd159959"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}