{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/customcrafter-customized-video-generation","title":"CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilities","arxiv_id":"2408.13239","date":"2024-08-23","proceeding":null,"authors":["Tao Wu","Yong Zhang","Xintao Wang","Xianpan Zhou","Guangcong Zheng","Zhongang Qi","Ying Shan","Xi Li"],"abstract":"Customized video generation aims to generate high-quality videos guided by text prompts and subject's reference images. However, since it is only trained on static images, the fine-tuning process of subject learning disrupts abilities of video diffusion models (VDMs) to combine concepts and generate motions. To restore these abilities, some methods use additional video similar to the prompt to fine-tune or guide the model. This requires frequent changes of guiding videos and even re-tuning of the model when generating different motions, which is very inconvenient for users. In this paper, we propose CustomCrafter, a novel framework that preserves the model's motion generation and conceptual combination abilities without additional video and fine-tuning to recovery. For preserving conceptual combination ability, we design a plug-and-play module to update few parameters in VDMs, enhancing the model's ability to capture the appearance details and the ability of concept combinations for new subjects. For motion generation, we observed that VDMs tend to restore the motion of video in the early stage of denoising, while focusing on the recovery of subject details in the later stage. Therefore, we propose Dynamic Weighted Video Sampling Strategy. Using the pluggability of our subject learning modules, we reduce the impact of this module on motion generation in the early stage of denoising, preserving the ability to generate motion of VDMs. In the later stage of denoising, we restore this module to repair the appearance details of the specified subject, thereby ensuring the fidelity of the subject's appearance. Experimental results show that our method has a significant improvement compared to previous methods. Code is available at https://github.com/WuTao-CS/CustomCrafter","url_abs":"https://arxiv.org/abs/2408.13239v2","url_pdf":"https://arxiv.org/pdf/2408.13239v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"customcrafter-customized-video-generation","repo_url":"https://github.com/wutao-cs/customcrafter","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"motion-generation","task_name":"Motion Generation"},{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2408.13239","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.13239"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wutao-cs/customcrafter","reach":{"status":"ok"}}],"summary":{"ran":7,"unverified":3},"by_repo_kind":{"official":{"samples":10,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"57bc51d1258b1398","entry":"createConvFunc","repo":"wutao-cs/customcrafter","repo_kind":"official","path":"extralibs/model_edge.py","file_url":"https://github.com/wutao-cs/customcrafter/blob/HEAD/extralibs/model_edge.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"57bc51d1258b1398"}},{"code_sha256_prefix":"a6702654efad4537","entry":"dynamic_degree","repo":"wutao-cs/customcrafter","repo_kind":"official","path":"eval/eval_dd.py","file_url":"https://github.com/wutao-cs/customcrafter/blob/HEAD/eval/eval_dd.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a6702654efad4537"}},{"code_sha256_prefix":"0e67fae78e6dd37c","entry":"format_time","repo":"wutao-cs/customcrafter","repo_kind":"official","path":"eval/dnnlib/util.py","file_url":"https://github.com/wutao-cs/customcrafter/blob/HEAD/eval/dnnlib/util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0e67fae78e6dd37c"}},{"code_sha256_prefix":"9fb4f79f8c4dc719","entry":"get_filelist","repo":"wutao-cs/customcrafter","repo_kind":"official","path":"eval/eval_fvd_kvd.py","file_url":"https://github.com/wutao-cs/customcrafter/blob/HEAD/eval/eval_fvd_kvd.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9fb4f79f8c4dc719"}},{"code_sha256_prefix":"8b5cf3d00508f17b","entry":"img2tensor","repo":"wutao-cs/customcrafter","repo_kind":"official","path":"extralibs/model_edge.py","file_url":"https://github.com/wutao-cs/customcrafter/blob/HEAD/extralibs/model_edge.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8b5cf3d00508f17b"}},{"code_sha256_prefix":"fbe06db04a182567","entry":"read_prompt_from_txt","repo":"wutao-cs/customcrafter","repo_kind":"official","path":"eval/eval_clip.py","file_url":"https://github.com/wutao-cs/customcrafter/blob/HEAD/eval/eval_clip.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fbe06db04a182567"}},{"code_sha256_prefix":"900a8d68bf48579e","entry":"tuple_product","repo":"wutao-cs/customcrafter","repo_kind":"official","path":"eval/dnnlib/util.py","file_url":"https://github.com/wutao-cs/customcrafter/blob/HEAD/eval/dnnlib/util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"900a8d68bf48579e"}},{"code_sha256_prefix":"9d31d2c4cd16bb2d","entry":"ask_yes_no","repo":"wutao-cs/customcrafter","repo_kind":"official","path":"eval/dnnlib/util.py","file_url":"https://github.com/wutao-cs/customcrafter/blob/HEAD/eval/dnnlib/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9d31d2c4cd16bb2d"}},{"code_sha256_prefix":"7c2f23c3b58f25a1","entry":"config_model","repo":"wutao-cs/customcrafter","repo_kind":"official","path":"extralibs/model_edge.py","file_url":"https://github.com/wutao-cs/customcrafter/blob/HEAD/extralibs/model_edge.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7c2f23c3b58f25a1"}},{"code_sha256_prefix":"87fbedd0c27cf378","entry":"nondefault_trainer_args","repo":"wutao-cs/customcrafter","repo_kind":"official","path":"custom.py","file_url":"https://github.com/wutao-cs/customcrafter/blob/HEAD/custom.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"87fbedd0c27cf378"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}