{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/caphuman-capture-your-moments-in-parallel","title":"CapHuman: Capture Your Moments in Parallel Universes","arxiv_id":"2402.00627","date":"2024-02-01","proceeding":"CVPR 2024 1","authors":["Chao Liang","Fan Ma","Linchao Zhu","Yingying Deng","Yi Yang"],"abstract":"We concentrate on a novel human-centric image synthesis task, that is, given only one reference facial photograph, it is expected to generate specific individual images with diverse head positions, poses, facial expressions, and illuminations in different contexts. To accomplish this goal, we argue that our generative model should be capable of the following favorable characteristics: (1) a strong visual and semantic understanding of our world and human society for basic object and human image generation. (2) generalizable identity preservation ability. (3) flexible and fine-grained head control. Recently, large pre-trained text-to-image diffusion models have shown remarkable results, serving as a powerful generative foundation. As a basis, we aim to unleash the above two capabilities of the pre-trained model. In this work, we present a new framework named CapHuman. We embrace the \"encode then learn to align\" paradigm, which enables generalizable identity preservation for new individuals without cumbersome tuning at inference. CapHuman encodes identity features and then learns to align them into the latent space. Moreover, we introduce the 3D facial prior to equip our model with control over the human head in a flexible and 3D-consistent manner. Extensive qualitative and quantitative analyses demonstrate our CapHuman can produce well-identity-preserved, photo-realistic, and high-fidelity portraits with content-rich representations and various head renditions, superior to established baselines. Code and checkpoint will be released at https://github.com/VamosC/CapHuman.","url_abs":"https://arxiv.org/abs/2402.00627v3","url_pdf":"https://arxiv.org/pdf/2402.00627v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"caphuman-capture-your-moments-in-parallel","repo_url":"https://github.com/vamosc/caphuman","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2402.00627","atlas_url":"https://app.syntology.ai/?focus=2402.00627","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.00627"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vamosc/caphuman","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_draft_wrong":3,"ran_violates":2,"ran":3,"unverified":2},"by_repo_kind":{"official":{"samples":10,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"vamosc/caphuman","repo_kind":"official","path":"libs/face_parsing/resnet.py","file_url":"https://github.com/vamosc/caphuman/blob/HEAD/libs/face_parsing/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"325b8d54f52e2421","entry":"create_layer_basic","repo":"vamosc/caphuman","repo_kind":"official","path":"libs/face_parsing/resnet.py","file_url":"https://github.com/vamosc/caphuman/blob/HEAD/libs/face_parsing/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"325b8d54f52e2421"}},{"code_sha256_prefix":"424012cb37b31172","entry":"default","repo":"vamosc/caphuman","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/vamosc/caphuman/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"424012cb37b31172"}},{"code_sha256_prefix":"0ea9678657d41123","entry":"draw_facepose","repo":"vamosc/caphuman","repo_kind":"official","path":"inference.py","file_url":"https://github.com/vamosc/caphuman/blob/HEAD/inference.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"0ea9678657d41123"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"vamosc/caphuman","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/vamosc/caphuman/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"b1368330cf0f5642","entry":"isimage","repo":"vamosc/caphuman","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/vamosc/caphuman/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b1368330cf0f5642"}},{"code_sha256_prefix":"d72762b700feee6f","entry":"ismap","repo":"vamosc/caphuman","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/vamosc/caphuman/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d72762b700feee6f"}},{"code_sha256_prefix":"9a299fe5ae09e407","entry":"uniq","repo":"vamosc/caphuman","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/vamosc/caphuman/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9a299fe5ae09e407"}},{"code_sha256_prefix":"c5bcc31f1c74ae5c","entry":"get_id_feat","repo":"vamosc/caphuman","repo_kind":"official","path":"inference.py","file_url":"https://github.com/vamosc/caphuman/blob/HEAD/inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c5bcc31f1c74ae5c"}},{"code_sha256_prefix":"6b4c1a8ce5fb7282","entry":"log_txt_as_img","repo":"vamosc/caphuman","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/vamosc/caphuman/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6b4c1a8ce5fb7282"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}