{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bivdiff-a-training-free-framework-for-general","title":"BIVDiff: A Training-Free Framework for General-Purpose Video Synthesis via Bridging Image and Video Diffusion Models","arxiv_id":"2312.02813","date":"2023-12-05","proceeding":"CVPR 2024 1","authors":["Fengyuan Shi","Jiaxi Gu","Hang Xu","Songcen Xu","Wei zhang","LiMin Wang"],"abstract":"Diffusion models have made tremendous progress in text-driven image and video generation. Now text-to-image foundation models are widely applied to various downstream image synthesis tasks, such as controllable image generation and image editing, while downstream video synthesis tasks are less explored for several reasons. First, it requires huge memory and computation overhead to train a video generation foundation model. Even with video foundation models, additional costly training is still required for downstream video synthesis tasks. Second, although some works extend image diffusion models into videos in a training-free manner, temporal consistency cannot be well preserved. Finally, these adaption methods are specifically designed for one task and fail to generalize to different tasks. To mitigate these issues, we propose a training-free general-purpose video synthesis framework, coined as {\\bf BIVDiff}, via bridging specific image diffusion models and general text-to-video foundation diffusion models. Specifically, we first use a specific image diffusion model (e.g., ControlNet and Instruct Pix2Pix) for frame-wise video generation, then perform Mixed Inversion on the generated video, and finally input the inverted latents into the video diffusion models (e.g., VidRD and ZeroScope) for temporal smoothing. This decoupled framework enables flexible image model selection for different purposes with strong task generalization and high efficiency. To validate the effectiveness and general use of BIVDiff, we perform a wide range of video synthesis tasks, including controllable video generation, video editing, video inpainting, and outpainting.","url_abs":"https://arxiv.org/abs/2312.02813v2","url_pdf":"https://arxiv.org/pdf/2312.02813v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bivdiff-a-training-free-framework-for-general","repo_url":"https://github.com/mcg-nju/bivdiff","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"model-selection","task_name":"Model Selection"},{"task_slug":"video-editing","task_name":"Video Editing"},{"task_slug":"video-generation","task_name":"Video Generation"},{"task_slug":"video-inpainting","task_name":"Video Inpainting"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"pixel-prediction","method_name":"Inpainting"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.02813","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.02813"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mcg-nju/bivdiff","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_honours":1,"ran_draft_wrong":1,"ran":1,"unverified":3},"by_repo_kind":{"official":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4b1e4330d47f40f3","entry":"betas_for_alpha_bar","repo":"mcg-nju/bivdiff","repo_kind":"official","path":"models/scheduling_ddim.py","file_url":"https://github.com/mcg-nju/bivdiff/blob/HEAD/models/scheduling_ddim.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4b1e4330d47f40f3"}},{"code_sha256_prefix":"bea2d776a332f2b0","entry":"rescale_noise_cfg","repo":"mcg-nju/bivdiff","repo_kind":"official","path":"models/Prompt2Prompt/pipeline_prompt2prompt.py","file_url":"https://github.com/mcg-nju/bivdiff/blob/HEAD/models/Prompt2Prompt/pipeline_prompt2prompt.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bea2d776a332f2b0"}},{"code_sha256_prefix":"f55cc06c1f75e2b8","entry":"update_alpha_time_word","repo":"mcg-nju/bivdiff","repo_kind":"official","path":"models/Prompt2Prompt/pipeline_prompt2prompt.py","file_url":"https://github.com/mcg-nju/bivdiff/blob/HEAD/models/Prompt2Prompt/pipeline_prompt2prompt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f55cc06c1f75e2b8"}},{"code_sha256_prefix":"3a3a0a7b84790627","entry":"create_controller","repo":"mcg-nju/bivdiff","repo_kind":"official","path":"models/Prompt2Prompt/pipeline_prompt2prompt.py","file_url":"https://github.com/mcg-nju/bivdiff/blob/HEAD/models/Prompt2Prompt/pipeline_prompt2prompt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3a3a0a7b84790627"}},{"code_sha256_prefix":"4f3220902109eef8","entry":"infer_prompt2prompt","repo":"mcg-nju/bivdiff","repo_kind":"official","path":"models/Prompt2Prompt/infer.py","file_url":"https://github.com/mcg-nju/bivdiff/blob/HEAD/models/Prompt2Prompt/infer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4f3220902109eef8"}},{"code_sha256_prefix":"a0a4d94af120c56f","entry":"init_prompt","repo":"mcg-nju/bivdiff","repo_kind":"official","path":"models/util.py","file_url":"https://github.com/mcg-nju/bivdiff/blob/HEAD/models/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a0a4d94af120c56f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}