{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/facial-synthesizing-dynamic-talking-face-with","title":"FACIAL: Synthesizing Dynamic Talking Face with Implicit Attribute Learning","arxiv_id":"2108.07938","date":"2021-08-18","proceeding":"ICCV 2021 10","authors":["Chenxu Zhang","Yifan Zhao","Yifei HUANG","Ming Zeng","Saifeng Ni","Madhukar Budagavi","Xiaohu Guo"],"abstract":"In this paper, we propose a talking face generation method that takes an audio signal as input and a short target video clip as reference, and synthesizes a photo-realistic video of the target face with natural lip motions, head poses, and eye blinks that are in-sync with the input audio signal. We note that the synthetic face attributes include not only explicit ones such as lip motions that have high correlations with speech, but also implicit ones such as head poses and eye blinks that have only weak correlation with the input audio. To model such complicated relationships among different face attributes with input audio, we propose a FACe Implicit Attribute Learning Generative Adversarial Network (FACIAL-GAN), which integrates the phonetics-aware, context-aware, and identity-aware information to synthesize the 3D face animation with realistic motions of lips, head poses, and eye blinks. Then, our Rendering-to-Video network takes the rendered face images and the attention map of eye blinks as input to generate the photo-realistic output video frames. Experimental results and user studies show our method can generate realistic talking face videos with not only synchronized lip motions, but also natural head movements and eye blinks, with better qualities than the results of state-of-the-art methods.","url_abs":"https://arxiv.org/abs/2108.07938v1","url_pdf":"https://arxiv.org/pdf/2108.07938v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"facial-synthesizing-dynamic-talking-face-with","repo_url":"https://github.com/zhangchenxu528/FACIAL","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3d-face-animation","task_name":"3D Face Animation"},{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"face-generation","task_name":"Face Generation"},{"task_slug":null,"task_name":"Generative Adversarial Network"},{"task_slug":"talking-face-generation","task_name":"Talking Face Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2108.07938","atlas_url":"https://app.syntology.ai/?focus=2108.07938","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2108.07938"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhangchenxu528/FACIAL","reach":null}],"summary":{"ran":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"a3134b134d8798d8","entry":"TfaceGAN","repo":"zhangchenxu528/FACIAL","repo_kind":"official","path":"audio2face/model.py","file_url":"https://github.com/zhangchenxu528/FACIAL/blob/HEAD/audio2face/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"a3134b134d8798d8"}},{"code_sha256_prefix":"f13f51d8a9ad1afa","entry":"UnetSkipConnectionBlock","repo":"zhangchenxu528/FACIAL","repo_kind":"official","path":"audio2face/model.py","file_url":"https://github.com/zhangchenxu528/FACIAL/blob/HEAD/audio2face/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"f13f51d8a9ad1afa"}},{"code_sha256_prefix":"095cc1bc69389008","entry":"zcxNet","repo":"zhangchenxu528/FACIAL","repo_kind":"official","path":"audio2face/model.py","file_url":"https://github.com/zhangchenxu528/FACIAL/blob/HEAD/audio2face/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"095cc1bc69389008"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}