{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dpe-disentanglement-of-pose-and-expression","title":"DPE: Disentanglement of Pose and Expression for General Video Portrait Editing","arxiv_id":"2301.06281","date":"2023-01-16","proceeding":"CVPR 2023 1","authors":["Youxin Pang","Yong Zhang","Weize Quan","Yanbo Fan","Xiaodong Cun","Ying Shan","Dong-Ming Yan"],"abstract":"One-shot video-driven talking face generation aims at producing a synthetic talking video by transferring the facial motion from a video to an arbitrary portrait image. Head pose and facial expression are always entangled in facial motion and transferred simultaneously. However, the entanglement sets up a barrier for these methods to be used in video portrait editing directly, where it may require to modify the expression only while maintaining the pose unchanged. One challenge of decoupling pose and expression is the lack of paired data, such as the same pose but different expressions. Only a few methods attempt to tackle this challenge with the feat of 3D Morphable Models (3DMMs) for explicit disentanglement. But 3DMMs are not accurate enough to capture facial details due to the limited number of Blenshapes, which has side effects on motion transfer. In this paper, we introduce a novel self-supervised disentanglement framework to decouple pose and expression without 3DMMs and paired data, which consists of a motion editing module, a pose generator, and an expression generator. The editing module projects faces into a latent space where pose motion and expression motion can be disentangled, and the pose or expression transfer can be performed in the latent space conveniently via addition. The two generators render the modified latent codes to images, respectively. Moreover, to guarantee the disentanglement, we propose a bidirectional cyclic training strategy with well-designed constraints. Evaluations demonstrate our method can control pose or expression independently and be used for general video editing.","url_abs":"https://arxiv.org/abs/2301.06281v2","url_pdf":"https://arxiv.org/pdf/2301.06281v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dpe-disentanglement-of-pose-and-expression","repo_url":"https://github.com/Carlyx/DPE","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"disentanglement","task_name":"Disentanglement"},{"task_slug":"face-generation","task_name":"Face Generation"},{"task_slug":"talking-face-generation","task_name":"Talking Face Generation"},{"task_slug":"video-editing","task_name":"Video Editing"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2301.06281","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2301.06281"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Carlyx/DPE","reach":null}],"summary":{"ran":11,"ran_draft_wrong":1,"ran_violates":1,"ran_fixture":2,"unverified":8},"by_repo_kind":{"official":{"samples":22,"ran":14,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"d56e5456e614facc","entry":"Blur","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d56e5456e614facc"}},{"code_sha256_prefix":"2791e7e6779bf494","entry":"ConstantInput","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2791e7e6779bf494"}},{"code_sha256_prefix":"9f3db3d59434def1","entry":"Direction","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f3db3d59434def1"}},{"code_sha256_prefix":"48671646da78ef35","entry":"EncoderApp","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"48671646da78ef35"}},{"code_sha256_prefix":"56cc85f0f1ca1e37","entry":"EqualConv2d","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"56cc85f0f1ca1e37"}},{"code_sha256_prefix":"ada97d5a14aed1a7","entry":"EqualLinear","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ada97d5a14aed1a7"}},{"code_sha256_prefix":"1bfc9df1f20629ae","entry":"FusedLeakyReLU","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1bfc9df1f20629ae"}},{"code_sha256_prefix":"fbc7a646183a73f6","entry":"ModulatedConv2d","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fbc7a646183a73f6"}},{"code_sha256_prefix":"a803ab12c3986ffc","entry":"NoiseInjection","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a803ab12c3986ffc"}},{"code_sha256_prefix":"bd5e20e1864404e7","entry":"ScaledLeakyReLU","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bd5e20e1864404e7"}},{"code_sha256_prefix":"3cb4513942fd17d4","entry":"Upsample","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3cb4513942fd17d4"}},{"code_sha256_prefix":"44c69c91b5434020","entry":"fused_leaky_relu","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"44c69c91b5434020"}},{"code_sha256_prefix":"6f65e378a4313f87","entry":"make_kernel","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"6f65e378a4313f87"}},{"code_sha256_prefix":"c5d084250637f1f8","entry":"upfirdn2d","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c5d084250637f1f8"}},{"code_sha256_prefix":"200baf9e83c9a4de","entry":"upfirdn2d_native","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"200baf9e83c9a4de"}},{"code_sha256_prefix":"8c8d2d8103bf9805","entry":"ConvLayer","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8c8d2d8103bf9805"}},{"code_sha256_prefix":"52dad034d993c841","entry":"Encoder","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"52dad034d993c841"}},{"code_sha256_prefix":"aaeb71b262fddbbc","entry":"Generator","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aaeb71b262fddbbc"}},{"code_sha256_prefix":"8db5aa5bd9cfcd07","entry":"ResBlock","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8db5aa5bd9cfcd07"}},{"code_sha256_prefix":"fe5a013ac94efe7a","entry":"StyledConv","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe5a013ac94efe7a"}},{"code_sha256_prefix":"0a86e04f7ae8efc1","entry":"Synthesis","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0a86e04f7ae8efc1"}},{"code_sha256_prefix":"4a62baba6f413f56","entry":"ToFlow","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4a62baba6f413f56"}},{"code_sha256_prefix":"a394679999e6ea1c","entry":"ToRGB","repo":"Carlyx/DPE","repo_kind":"official","path":"networks/generator.py","file_url":"https://github.com/Carlyx/DPE/blob/HEAD/networks/generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a394679999e6ea1c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}