{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/shmt-self-supervised-hierarchical-makeup","title":"SHMT: Self-supervised Hierarchical Makeup Transfer via Latent Diffusion Models","arxiv_id":"2412.11058","date":"2024-12-15","proceeding":null,"authors":["Zhaoyang Sun","Shengwu Xiong","Yaxiong Chen","Fei Du","Weihua Chen","Fan Wang","Yi Rong"],"abstract":"This paper studies the challenging task of makeup transfer, which aims to apply diverse makeup styles precisely and naturally to a given facial image. Due to the absence of paired data, current methods typically synthesize sub-optimal pseudo ground truths to guide the model training, resulting in low makeup fidelity. Additionally, different makeup styles generally have varying effects on the person face, but existing methods struggle to deal with this diversity. To address these issues, we propose a novel Self-supervised Hierarchical Makeup Transfer (SHMT) method via latent diffusion models. Following a \"decoupling-and-reconstruction\" paradigm, SHMT works in a self-supervised manner, freeing itself from the misguidance of imprecise pseudo-paired data. Furthermore, to accommodate a variety of makeup styles, hierarchical texture details are decomposed via a Laplacian pyramid and selectively introduced to the content representation. Finally, we design a novel Iterative Dual Alignment (IDA) module that dynamically adjusts the injection condition of the diffusion model, allowing the alignment errors caused by the domain gap between content and makeup representations to be corrected. Extensive quantitative and qualitative analyses demonstrate the effectiveness of our method. Our code is available at \\url{https://github.com/Snowfallingplum/SHMT}.","url_abs":"https://arxiv.org/abs/2412.11058v1","url_pdf":"https://arxiv.org/pdf/2412.11058v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"shmt-self-supervised-hierarchical-makeup","repo_url":"https://github.com/snowfallingplum/shmt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"shmt-self-supervised-hierarchical-makeup","repo_url":"https://github.com/snowfallingplum/csd-mt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"laplacian-pyramid","method_name":"Laplacian Pyramid"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2412.11058","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.11058"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Snowfallingplum/SHMT","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_draft_wrong":3,"ran_violates":3,"ran":2,"unverified":2},"by_repo_kind":{"official":{"samples":10,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"fe5dd5258046898c","entry":"always","repo":"Snowfallingplum/SHMT","repo_kind":"official","path":"ldm/modules/x_transformer.py","file_url":"https://github.com/Snowfallingplum/SHMT/blob/HEAD/ldm/modules/x_transformer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fe5dd5258046898c"}},{"code_sha256_prefix":"8241c0562bc710fd","entry":"chunk","repo":"Snowfallingplum/SHMT","repo_kind":"official","path":"makeup_inference_model2.py","file_url":"https://github.com/Snowfallingplum/SHMT/blob/HEAD/makeup_inference_model2.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8241c0562bc710fd"}},{"code_sha256_prefix":"424012cb37b31172","entry":"default","repo":"Snowfallingplum/SHMT","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/Snowfallingplum/SHMT/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"424012cb37b31172"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"Snowfallingplum/SHMT","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/Snowfallingplum/SHMT/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"b1368330cf0f5642","entry":"isimage","repo":"Snowfallingplum/SHMT","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/Snowfallingplum/SHMT/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b1368330cf0f5642"}},{"code_sha256_prefix":"d72762b700feee6f","entry":"ismap","repo":"Snowfallingplum/SHMT","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/Snowfallingplum/SHMT/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d72762b700feee6f"}},{"code_sha256_prefix":"1e63d588563eb90a","entry":"numpy_to_pil","repo":"Snowfallingplum/SHMT","repo_kind":"official","path":"makeup_inference_model2.py","file_url":"https://github.com/Snowfallingplum/SHMT/blob/HEAD/makeup_inference_model2.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1e63d588563eb90a"}},{"code_sha256_prefix":"9a299fe5ae09e407","entry":"uniq","repo":"Snowfallingplum/SHMT","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/Snowfallingplum/SHMT/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9a299fe5ae09e407"}},{"code_sha256_prefix":"4533e43e06a12dc5","entry":"download_models","repo":"Snowfallingplum/SHMT","repo_kind":"official","path":"notebook_helpers.py","file_url":"https://github.com/Snowfallingplum/SHMT/blob/HEAD/notebook_helpers.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4533e43e06a12dc5"}},{"code_sha256_prefix":"f9bd2e83191afad1","entry":"log_txt_as_img","repo":"Snowfallingplum/SHMT","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/Snowfallingplum/SHMT/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f9bd2e83191afad1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}