{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/freeinit-bridging-initialization-gap-in-video","title":"FreeInit: Bridging Initialization Gap in Video Diffusion Models","arxiv_id":"2312.07537","date":"2023-12-12","proceeding":null,"authors":["Tianxing Wu","Chenyang Si","Yuming Jiang","Ziqi Huang","Ziwei Liu"],"abstract":"Though diffusion-based video generation has witnessed rapid progress, the inference results of existing models still exhibit unsatisfactory temporal consistency and unnatural dynamics. In this paper, we delve deep into the noise initialization of video diffusion models, and discover an implicit training-inference gap that attributes to the unsatisfactory inference quality.Our key findings are: 1) the spatial-temporal frequency distribution of the initial noise at inference is intrinsically different from that for training, and 2) the denoising process is significantly influenced by the low-frequency components of the initial noise. Motivated by these observations, we propose a concise yet effective inference sampling strategy, FreeInit, which significantly improves temporal consistency of videos generated by diffusion models. Through iteratively refining the spatial-temporal low-frequency components of the initial latent during inference, FreeInit is able to compensate the initialization gap between training and inference, thus effectively improving the subject appearance and temporal consistency of generation results. Extensive experiments demonstrate that FreeInit consistently enhances the generation quality of various text-to-video diffusion models without additional training or fine-tuning.","url_abs":"https://arxiv.org/abs/2312.07537v2","url_pdf":"https://arxiv.org/pdf/2312.07537v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"freeinit-bridging-initialization-gap-in-video","repo_url":"https://github.com/tianxingwu/freeinit","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"text-to-video-generation","task_name":"Text-to-Video Generation"},{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.07537","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.07537"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tianxingwu/freeinit","reach":null}],"summary":{"ran_fixture":2,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e285f4a541803f96","entry":"freq_mix_3d","repo":"tianxingwu/freeinit","repo_kind":"official","path":"examples/AnimateDiff/animatediff/utils/freeinit_utils.py","file_url":"https://github.com/tianxingwu/freeinit/blob/HEAD/examples/AnimateDiff/animatediff/utils/freeinit_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e285f4a541803f96"}},{"code_sha256_prefix":"5753a1ae1b2b2a7a","entry":"gaussian_low_pass_filter","repo":"tianxingwu/freeinit","repo_kind":"official","path":"examples/AnimateDiff/animatediff/utils/freeinit_utils.py","file_url":"https://github.com/tianxingwu/freeinit/blob/HEAD/examples/AnimateDiff/animatediff/utils/freeinit_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5753a1ae1b2b2a7a"}},{"code_sha256_prefix":"e6e9076eaee4d40e","entry":"get_freq_filter","repo":"tianxingwu/freeinit","repo_kind":"official","path":"examples/AnimateDiff/animatediff/utils/freeinit_utils.py","file_url":"https://github.com/tianxingwu/freeinit/blob/HEAD/examples/AnimateDiff/animatediff/utils/freeinit_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e6e9076eaee4d40e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}