{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/animatediff-animate-your-personalized-text-to","title":"AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning","arxiv_id":"2307.04725","date":"2023-07-10","proceeding":null,"authors":["Yuwei Guo","Ceyuan Yang","Anyi Rao","Zhengyang Liang","Yaohui Wang","Yu Qiao","Maneesh Agrawala","Dahua Lin","Bo Dai"],"abstract":"With the advance of text-to-image (T2I) diffusion models (e.g., Stable Diffusion) and corresponding personalization techniques such as DreamBooth and LoRA, everyone can manifest their imagination into high-quality images at an affordable cost. However, adding motion dynamics to existing high-quality personalized T2Is and enabling them to generate animations remains an open challenge. In this paper, we present AnimateDiff, a practical framework for animating personalized T2I models without requiring model-specific tuning. At the core of our framework is a plug-and-play motion module that can be trained once and seamlessly integrated into any personalized T2Is originating from the same base T2I. Through our proposed training strategy, the motion module effectively learns transferable motion priors from real-world videos. Once trained, the motion module can be inserted into a personalized T2I model to form a personalized animation generator. We further propose MotionLoRA, a lightweight fine-tuning technique for AnimateDiff that enables a pre-trained motion module to adapt to new motion patterns, such as different shot types, at a low training and data collection cost. We evaluate AnimateDiff and MotionLoRA on several public representative personalized T2I models collected from the community. The results demonstrate that our approaches help these models generate temporally smooth animation clips while preserving the visual quality and motion diversity. Codes and pre-trained weights are available at https://github.com/guoyww/AnimateDiff.","url_abs":"https://arxiv.org/abs/2307.04725v2","url_pdf":"https://arxiv.org/pdf/2307.04725v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"animatediff-animate-your-personalized-text-to","repo_url":"https://github.com/guoyww/animatediff","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"animatediff-animate-your-personalized-text-to","repo_url":"https://github.com/arthur-qiu/FreeNoise-AnimateDiff","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"animatediff-animate-your-personalized-text-to","repo_url":"https://github.com/bujiazi/motionclone","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"animatediff-animate-your-personalized-text-to","repo_url":"https://github.com/lpengyang/motionclone","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"animatediff-animate-your-personalized-text-to","repo_url":"https://github.com/snap-research/panda-70m","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"animatediff-animate-your-personalized-text-to","repo_url":"https://github.com/sunzey/alphaclip","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"animatediff-animate-your-personalized-text-to","repo_url":"https://github.com/talesofai/animatediff","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"animatediff-animate-your-personalized-text-to","repo_url":"https://github.com/yhZhai/mcm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-animation","task_name":"Image Animation"}],"methods":[{"method_slug":"base","method_name":"BASE"},{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2307.04725","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.04725"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/arthur-qiu/FreeNoise-AnimateDiff","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sunzey/alphaclip","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yhZhai/mcm","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lpengyang/motionclone","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bujiazi/motionclone","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/talesofai/animatediff","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/snap-research/panda-70m","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/guoyww/animatediff","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":2,"ran_draft_wrong":1,"unverified":7},"by_repo_kind":{"listed":{"samples":10,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0b088617d5525751","entry":"get_noise_pred_single","repo":"talesofai/animatediff","repo_kind":"listed","path":"animatediff/utils/util.py","file_url":"https://github.com/talesofai/animatediff/blob/HEAD/animatediff/utils/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0b088617d5525751"}},{"code_sha256_prefix":"fb032d6fa7e6196d","entry":"next_step","repo":"talesofai/animatediff","repo_kind":"listed","path":"animatediff/utils/util.py","file_url":"https://github.com/talesofai/animatediff/blob/HEAD/animatediff/utils/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fb032d6fa7e6196d"}},{"code_sha256_prefix":"4719c763c53be3fe","entry":"zero_module","repo":"talesofai/animatediff","repo_kind":"listed","path":"animatediff/models/motion_module.py","file_url":"https://github.com/talesofai/animatediff/blob/HEAD/animatediff/models/motion_module.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4719c763c53be3fe"}},{"code_sha256_prefix":"9736e3792b57e25c","entry":"convert_lora","repo":"talesofai/animatediff","repo_kind":"listed","path":"animatediff/utils/convert_lora_safetensor_to_diffusers.py","file_url":"https://github.com/talesofai/animatediff/blob/HEAD/animatediff/utils/convert_lora_safetensor_to_diffusers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9736e3792b57e25c"}},{"code_sha256_prefix":"8bcc66e888dc7b8a","entry":"get_down_block","repo":"talesofai/animatediff","repo_kind":"listed","path":"animatediff/models/unet_blocks.py","file_url":"https://github.com/talesofai/animatediff/blob/HEAD/animatediff/models/unet_blocks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8bcc66e888dc7b8a"}},{"code_sha256_prefix":"4ab05f5e735bedde","entry":"get_up_block","repo":"talesofai/animatediff","repo_kind":"listed","path":"animatediff/models/unet_blocks.py","file_url":"https://github.com/talesofai/animatediff/blob/HEAD/animatediff/models/unet_blocks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4ab05f5e735bedde"}},{"code_sha256_prefix":"a0a4d94af120c56f","entry":"init_prompt","repo":"talesofai/animatediff","repo_kind":"listed","path":"animatediff/utils/util.py","file_url":"https://github.com/talesofai/animatediff/blob/HEAD/animatediff/utils/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a0a4d94af120c56f"}},{"code_sha256_prefix":"dfb052a0af48540f","entry":"renew_resnet_paths","repo":"talesofai/animatediff","repo_kind":"listed","path":"animatediff/utils/convert_from_ckpt.py","file_url":"https://github.com/talesofai/animatediff/blob/HEAD/animatediff/utils/convert_from_ckpt.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dfb052a0af48540f"}},{"code_sha256_prefix":"c5ec9c13e456a046","entry":"renew_vae_resnet_paths","repo":"talesofai/animatediff","repo_kind":"listed","path":"animatediff/utils/convert_from_ckpt.py","file_url":"https://github.com/talesofai/animatediff/blob/HEAD/animatediff/utils/convert_from_ckpt.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c5ec9c13e456a046"}},{"code_sha256_prefix":"cea0bc82e0c96896","entry":"shave_segments","repo":"talesofai/animatediff","repo_kind":"listed","path":"animatediff/utils/convert_from_ckpt.py","file_url":"https://github.com/talesofai/animatediff/blob/HEAD/animatediff/utils/convert_from_ckpt.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cea0bc82e0c96896"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}