{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/videocrafter1-open-diffusion-models-for-high","title":"VideoCrafter1: Open Diffusion Models for High-Quality Video Generation","arxiv_id":"2310.19512","date":"2023-10-30","proceeding":null,"authors":["Haoxin Chen","Menghan Xia","Yingqing He","Yong Zhang","Xiaodong Cun","Shaoshu Yang","Jinbo Xing","Yaofang Liu","Qifeng Chen","Xintao Wang","Chao Weng","Ying Shan"],"abstract":"Video generation has increasingly gained interest in both academia and industry. Although commercial tools can generate plausible videos, there is a limited number of open-source models available for researchers and engineers. In this work, we introduce two diffusion models for high-quality video generation, namely text-to-video (T2V) and image-to-video (I2V) models. T2V models synthesize a video based on a given text input, while I2V models incorporate an additional image input. Our proposed T2V model can generate realistic and cinematic-quality videos with a resolution of $1024 \\times 576$, outperforming other open-source T2V models in terms of quality. The I2V model is designed to produce videos that strictly adhere to the content of the provided reference image, preserving its content, structure, and style. This model is the first open-source I2V foundation model capable of transforming a given image into a video clip while maintaining content preservation constraints. We believe that these open-source video generation models will contribute significantly to the technological advancements within the community.","url_abs":"https://arxiv.org/abs/2310.19512v1","url_pdf":"https://arxiv.org/pdf/2310.19512v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"videocrafter1-open-diffusion-models-for-high","repo_url":"https://github.com/ailab-cvc/videocrafter","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"videocrafter1-open-diffusion-models-for-high","repo_url":"https://github.com/invictus717/interactivevideo","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"videocrafter1-open-diffusion-models-for-high","repo_url":"https://github.com/videocrafter/videocrafter","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"text-to-video-generation","task_name":"Text-to-Video Generation"},{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[{"method_slug":"clip","method_name":"CLIP"},{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/text-to-video-generation-on-evalcrafter-text","task":"Text-to-Video Generation","dataset":"EvalCrafter Text-to-Video (ECTV) Dataset","model":"VideoCrafter1","rank_in_archive_order":3,"of":5,"metrics":{"Motion Quality":"60.85","Temporal Consistency":"55.89","Text-to-Video Alignment":"61.95","Total Score":"232","Visual Quality":"53.08"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.19512","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.19512"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/invictus717/interactivevideo","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ailab-cvc/videocrafter","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/videocrafter/videocrafter","reach":null}],"summary":{"ran":5,"ran_honours":1,"ran_draft_wrong":1,"unverified":5},"by_repo_kind":{"listed":{"samples":12,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5dd5ccc011afb9c3","entry":"denormalize","repo":"invictus717/interactivevideo","repo_kind":"listed","path":"models/streamdiffusion/image_utils.py","file_url":"https://github.com/invictus717/interactivevideo/blob/HEAD/models/streamdiffusion/image_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5dd5ccc011afb9c3"}},{"code_sha256_prefix":"35d471d9e81ee42a","entry":"is_installed","repo":"invictus717/interactivevideo","repo_kind":"listed","path":"models/streamdiffusion/pip_utils.py","file_url":"https://github.com/invictus717/interactivevideo/blob/HEAD/models/streamdiffusion/pip_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"35d471d9e81ee42a"}},{"code_sha256_prefix":"b188597984f52631","entry":"numpy_to_pil","repo":"invictus717/interactivevideo","repo_kind":"listed","path":"models/streamdiffusion/image_utils.py","file_url":"https://github.com/invictus717/interactivevideo/blob/HEAD/models/streamdiffusion/image_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b188597984f52631"}},{"code_sha256_prefix":"e0ca6f492a8f2cbf","entry":"pt_to_numpy","repo":"invictus717/interactivevideo","repo_kind":"listed","path":"models/streamdiffusion/image_utils.py","file_url":"https://github.com/invictus717/interactivevideo/blob/HEAD/models/streamdiffusion/image_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e0ca6f492a8f2cbf"}},{"code_sha256_prefix":"85ecb3cb5eeb8bc5","entry":"reverse_point_pairs","repo":"invictus717/interactivevideo","repo_kind":"listed","path":"demo/main_gradio.py","file_url":"https://github.com/invictus717/interactivevideo/blob/HEAD/demo/main_gradio.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"85ecb3cb5eeb8bc5"}},{"code_sha256_prefix":"635ea667a5c67443","entry":"version","repo":"invictus717/interactivevideo","repo_kind":"listed","path":"models/streamdiffusion/pip_utils.py","file_url":"https://github.com/invictus717/interactivevideo/blob/HEAD/models/streamdiffusion/pip_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"635ea667a5c67443"}},{"code_sha256_prefix":"4719c763c53be3fe","entry":"zero_module","repo":"invictus717/interactivevideo","repo_kind":"listed","path":"models/animatediff/models/motion_module.py","file_url":"https://github.com/invictus717/interactivevideo/blob/HEAD/models/animatediff/models/motion_module.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4719c763c53be3fe"}},{"code_sha256_prefix":"10970e3504f54097","entry":"draw_circle_on_mask","repo":"invictus717/interactivevideo","repo_kind":"listed","path":"demo/draw_utils.py","file_url":"https://github.com/invictus717/interactivevideo/blob/HEAD/demo/draw_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"10970e3504f54097"}},{"code_sha256_prefix":"9d38bb644aea4805","entry":"draw_mask_on_image","repo":"invictus717/interactivevideo","repo_kind":"listed","path":"demo/draw_utils.py","file_url":"https://github.com/invictus717/interactivevideo/blob/HEAD/demo/draw_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9d38bb644aea4805"}},{"code_sha256_prefix":"0ee47335b1dbc111","entry":"draw_points_on_image","repo":"invictus717/interactivevideo","repo_kind":"listed","path":"demo/draw_utils.py","file_url":"https://github.com/invictus717/interactivevideo/blob/HEAD/demo/draw_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0ee47335b1dbc111"}},{"code_sha256_prefix":"92a10aa9e4d73a7c","entry":"get_motion_module","repo":"invictus717/interactivevideo","repo_kind":"listed","path":"models/animatediff/models/motion_module.py","file_url":"https://github.com/invictus717/interactivevideo/blob/HEAD/models/animatediff/models/motion_module.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"92a10aa9e4d73a7c"}},{"code_sha256_prefix":"ef16b546127f1484","entry":"run_python","repo":"invictus717/interactivevideo","repo_kind":"listed","path":"models/streamdiffusion/pip_utils.py","file_url":"https://github.com/invictus717/interactivevideo/blob/HEAD/models/streamdiffusion/pip_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ef16b546127f1484"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}