{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/beyond-alignment-blind-video-face-restoration","title":"Beyond Alignment: Blind Video Face Restoration via Parsing-Guided Temporal-Coherent Transformer","arxiv_id":"2404.13640","date":"2024-04-21","proceeding":null,"authors":["Kepeng Xu","Li Xu","Gang He","Wenxin Yu","Yunsong Li"],"abstract":"Multiple complex degradations are coupled in low-quality video faces in the real world. Therefore, blind video face restoration is a highly challenging ill-posed problem, requiring not only hallucinating high-fidelity details but also enhancing temporal coherence across diverse pose variations. Restoring each frame independently in a naive manner inevitably introduces temporal incoherence and artifacts from pose changes and keypoint localization errors. To address this, we propose the first blind video face restoration approach with a novel parsing-guided temporal-coherent transformer (PGTFormer) without pre-alignment. PGTFormer leverages semantic parsing guidance to select optimal face priors for generating temporally coherent artifact-free results. Specifically, we pre-train a temporal-spatial vector quantized auto-encoder on high-quality video face datasets to extract expressive context-rich priors. Then, the temporal parse-guided codebook predictor (TPCP) restores faces in different poses based on face parsing context cues without performing face pre-alignment. This strategy reduces artifacts and mitigates jitter caused by cumulative errors from face pre-alignment. Finally, the temporal fidelity regulator (TFR) enhances fidelity through temporal feature interaction and improves video temporal consistency. Extensive experiments on face videos show that our method outperforms previous face restoration baselines. The code will be released on \\href{https://github.com/kepengxu/PGTFormer}{https://github.com/kepengxu/PGTFormer}.","url_abs":"https://arxiv.org/abs/2404.13640v1","url_pdf":"https://arxiv.org/pdf/2404.13640v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"beyond-alignment-blind-video-face-restoration","repo_url":"https://github.com/kepengxu/pgtformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"face-parsing","task_name":"Face Parsing"},{"task_slug":"semantic-parsing","task_name":"Semantic Parsing"},{"task_slug":"video-temporal-consistency","task_name":"Video Temporal Consistency"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2404.13640","atlas_url":"https://app.syntology.ai/?focus=2404.13640","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.13640"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/kepengxu/PGTFormer","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_draft_wrong":6,"ran":7,"ran_fixture":4,"unverified":2},"by_repo_kind":{"official":{"samples":19,"ran":17,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":19,"samples":[{"code_sha256_prefix":"c3a6b977022957cb","entry":"Normalize","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"archs/rqvae_arch.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/archs/rqvae_arch.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c3a6b977022957cb"}},{"code_sha256_prefix":"ad986575b3f8c295","entry":"adaptive_instance_normalization","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"archs/codeformer_arch.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/archs/codeformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"ad986575b3f8c295"}},{"code_sha256_prefix":"fac88a669d369ae6","entry":"calc_mean_std","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"archs/codeformer_arch.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/archs/codeformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"fac88a669d369ae6"}},{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"archs/pgtformer_arch.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/archs/pgtformer_arch.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"325b8d54f52e2421","entry":"create_layer_basic","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"archs/pgtformer_arch.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/archs/pgtformer_arch.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"325b8d54f52e2421"}},{"code_sha256_prefix":"aad0ea11cb13d353","entry":"get_window_size","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"modules/swin.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/modules/swin.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aad0ea11cb13d353"}},{"code_sha256_prefix":"fdbf43f49b676e7f","entry":"load_architecture","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"inference.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/inference.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"fdbf43f49b676e7f"}},{"code_sha256_prefix":"c302f98a4f537aa8","entry":"load_architecture","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"inference_cn.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/inference_cn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c302f98a4f537aa8"}},{"code_sha256_prefix":"c04a23de0531da4a","entry":"make_layer","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"modules/rstt_layers.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/modules/rstt_layers.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c04a23de0531da4a"}},{"code_sha256_prefix":"b8dc6e0bd98e6f35","entry":"nonlinearity","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"archs/rqvae_arch.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/archs/rqvae_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"b8dc6e0bd98e6f35"}},{"code_sha256_prefix":"c71d6184f8da9bac","entry":"normalize","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"archs/vqgan_arch.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/archs/vqgan_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c71d6184f8da9bac"}},{"code_sha256_prefix":"74742dff47341df1","entry":"rgbnp2tensor","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"easy_inference_in_windows.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/easy_inference_in_windows.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"74742dff47341df1"}},{"code_sha256_prefix":"107bf3b7b61a9085","entry":"swish","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"archs/pgtformer_arch.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/archs/pgtformer_arch.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"107bf3b7b61a9085"}},{"code_sha256_prefix":"c33b3de22ecbd051","entry":"window_partition","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"modules/swin.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/modules/swin.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c33b3de22ecbd051"}},{"code_sha256_prefix":"dbf81aece2a63747","entry":"window_partition","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"modules/rstt_layers.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/modules/rstt_layers.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dbf81aece2a63747"}},{"code_sha256_prefix":"ce51a79a418f8860","entry":"window_reverse","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"modules/swin.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/modules/swin.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ce51a79a418f8860"}},{"code_sha256_prefix":"15e141e09e3e5d8d","entry":"window_reverse","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"modules/rstt_layers.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/modules/rstt_layers.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"15e141e09e3e5d8d"}},{"code_sha256_prefix":"59291c0466ba9265","entry":"apply_net_to_frames","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"easy_inference_in_windows.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/easy_inference_in_windows.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"59291c0466ba9265"}},{"code_sha256_prefix":"7cf16126f70dcb9c","entry":"load_architecture","repo":"kepengxu/PGTFormer","repo_kind":"official","path":"easy_inference_in_windows.py","file_url":"https://github.com/kepengxu/PGTFormer/blob/HEAD/easy_inference_in_windows.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"7cf16126f70dcb9c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}