{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/depth-aware-generative-adversarial-network","title":"Depth-Aware Generative Adversarial Network for Talking Head Video Generation","arxiv_id":"2203.06605","date":"2022-03-13","proceeding":"CVPR 2022 1","authors":["Fa-Ting Hong","Longhao Zhang","Li Shen","Dan Xu"],"abstract":"Talking head video generation aims to produce a synthetic human face video that contains the identity and pose information respectively from a given source image and a driving video.Existing works for this task heavily rely on 2D representations (e.g. appearance and motion) learned from the input images. However, dense 3D facial geometry (e.g. pixel-wise depth) is extremely important for this task as it is particularly beneficial for us to essentially generate accurate 3D face structures and distinguish noisy information from the possibly cluttered background. Nevertheless, dense 3D geometry annotations are prohibitively costly for videos and are typically not available for this video generation task. In this paper, we first introduce a self-supervised geometry learning method to automatically recover the dense 3D geometry (i.e.depth) from the face videos without the requirement of any expensive 3D annotation data. Based on the learned dense depth maps, we further propose to leverage them to estimate sparse facial keypoints that capture the critical movement of the human head. In a more dense way, the depth is also utilized to learn 3D-aware cross-modal (i.e. appearance and depth) attention to guide the generation of motion fields for warping source image representations. All these contributions compose a novel depth-aware generative adversarial network (DaGAN) for talking head generation. Extensive experiments conducted demonstrate that our proposed method can generate highly realistic faces, and achieve significant results on the unseen human faces.","url_abs":"https://arxiv.org/abs/2203.06605v2","url_pdf":"https://arxiv.org/pdf/2203.06605v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"depth-aware-generative-adversarial-network","repo_url":"https://github.com/harlanhong/cvpr2022-dagan","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3d-geometry","task_name":"3D geometry"},{"task_slug":null,"task_name":"Generative Adversarial Network"},{"task_slug":"talking-head-generation","task_name":"Talking Head Generation"},{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.06605","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.06605"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/harlanhong/cvpr2022-dagan","reach":null}],"summary":{"ran":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"669f810ed9fa2307","entry":"DepthAwareAttention","repo":"harlanhong/cvpr2022-dagan","repo_kind":"official","path":"modules/generator.py","file_url":"https://github.com/harlanhong/cvpr2022-dagan/blob/HEAD/modules/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"669f810ed9fa2307"}},{"code_sha256_prefix":"80c43a66439237ca","entry":"DepthAwareGenerator","repo":"harlanhong/cvpr2022-dagan","repo_kind":"official","path":"modules/generator.py","file_url":"https://github.com/harlanhong/cvpr2022-dagan/blob/HEAD/modules/generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"80c43a66439237ca"}},{"code_sha256_prefix":"3285554fcbe73f00","entry":"ResBlock2d","repo":"harlanhong/cvpr2022-dagan","repo_kind":"official","path":"modules/generator.py","file_url":"https://github.com/harlanhong/cvpr2022-dagan/blob/HEAD/modules/generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3285554fcbe73f00"}},{"code_sha256_prefix":"47dc6bb6b92a9531","entry":"SameBlock2d","repo":"harlanhong/cvpr2022-dagan","repo_kind":"official","path":"modules/generator.py","file_url":"https://github.com/harlanhong/cvpr2022-dagan/blob/HEAD/modules/generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"47dc6bb6b92a9531"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}