{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/coarse-to-fine-latent-diffusion-for-pose","title":"Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesis","arxiv_id":"2402.18078","date":"2024-02-28","proceeding":"CVPR 2024 1","authors":["Yanzuo Lu","Manlin Zhang","Andy J Ma","Xiaohua Xie","Jian-Huang Lai"],"abstract":"Diffusion model is a promising approach to image generation and has been employed for Pose-Guided Person Image Synthesis (PGPIS) with competitive performance. While existing methods simply align the person appearance to the target pose, they are prone to overfitting due to the lack of a high-level semantic understanding on the source person image. In this paper, we propose a novel Coarse-to-Fine Latent Diffusion (CFLD) method for PGPIS. In the absence of image-caption pairs and textual prompts, we develop a novel training paradigm purely based on images to control the generation process of a pre-trained text-to-image diffusion model. A perception-refined decoder is designed to progressively refine a set of learnable queries and extract semantic understanding of person images as a coarse-grained prompt. This allows for the decoupling of fine-grained appearance and pose information controls at different stages, and thus circumventing the potential overfitting problem. To generate more realistic texture details, a hybrid-granularity attention module is proposed to encode multi-scale fine-grained appearance features as bias terms to augment the coarse-grained prompt. Both quantitative and qualitative experimental results on the DeepFashion benchmark demonstrate the superiority of our method over the state of the arts for PGPIS. Code is available at https://github.com/YanzuoLu/CFLD.","url_abs":"https://arxiv.org/abs/2402.18078v2","url_pdf":"https://arxiv.org/pdf/2402.18078v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"coarse-to-fine-latent-diffusion-for-pose","repo_url":"https://github.com/YanzuoLu/CFLD","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"pose-guided-image-generation","task_name":"Pose-Guided Image Generation"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.18078","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.18078"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/YanzuoLu/CFLD","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":1,"ran":2,"ran_violates":1,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"144d10b49baeb8a6","entry":"window_partition","repo":"YanzuoLu/CFLD","repo_kind":"official","path":"models/swin_transformer.py","file_url":"https://github.com/YanzuoLu/CFLD/blob/HEAD/models/swin_transformer.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":2,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"144d10b49baeb8a6"}},{"code_sha256_prefix":"ad890d9791b6be92","entry":"draw_pose_from_cords","repo":"YanzuoLu/CFLD","repo_kind":"official","path":"pose_utils.py","file_url":"https://github.com/YanzuoLu/CFLD/blob/HEAD/pose_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad890d9791b6be92"}},{"code_sha256_prefix":"ab4109634b75ef8b","entry":"is_image_file","repo":"YanzuoLu/CFLD","repo_kind":"official","path":"generate_fashion_datasets.py","file_url":"https://github.com/YanzuoLu/CFLD/blob/HEAD/generate_fashion_datasets.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ab4109634b75ef8b"}},{"code_sha256_prefix":"bfeedb348ae483c2","entry":"load_pose_cords_from_strings","repo":"YanzuoLu/CFLD","repo_kind":"official","path":"pose_utils.py","file_url":"https://github.com/YanzuoLu/CFLD/blob/HEAD/pose_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bfeedb348ae483c2"}},{"code_sha256_prefix":"61bf152e6a42a184","entry":"window_reverse","repo":"YanzuoLu/CFLD","repo_kind":"official","path":"models/swin_transformer.py","file_url":"https://github.com/YanzuoLu/CFLD/blob/HEAD/models/swin_transformer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61bf152e6a42a184"}},{"code_sha256_prefix":"0ffb0077d629ab71","entry":"build_backbone","repo":"YanzuoLu/CFLD","repo_kind":"official","path":"models/swin_transformer.py","file_url":"https://github.com/YanzuoLu/CFLD/blob/HEAD/models/swin_transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ffb0077d629ab71"}},{"code_sha256_prefix":"4b8d89aaca3cbec8","entry":"cords_to_map","repo":"YanzuoLu/CFLD","repo_kind":"official","path":"pose_utils.py","file_url":"https://github.com/YanzuoLu/CFLD/blob/HEAD/pose_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4b8d89aaca3cbec8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}