{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/consisti2v-enhancing-visual-consistency-for","title":"ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation","arxiv_id":"2402.04324","date":"2024-02-06","proceeding":null,"authors":["Weiming Ren","Huan Yang","Ge Zhang","Cong Wei","Xinrun Du","Wenhao Huang","Wenhu Chen"],"abstract":"Image-to-video (I2V) generation aims to use the initial frame (alongside a text prompt) to create a video sequence. A grand challenge in I2V generation is to maintain visual consistency throughout the video: existing methods often struggle to preserve the integrity of the subject, background, and style from the first frame, as well as ensure a fluid and logical progression within the video narrative. To mitigate these issues, we propose ConsistI2V, a diffusion-based method to enhance visual consistency for I2V generation. Specifically, we introduce (1) spatiotemporal attention over the first frame to maintain spatial and motion consistency, (2) noise initialization from the low-frequency band of the first frame to enhance layout consistency. These two approaches enable ConsistI2V to generate highly consistent videos. We also extend the proposed approaches to show their potential to improve consistency in auto-regressive long video generation and camera motion control. To verify the effectiveness of our method, we propose I2V-Bench, a comprehensive evaluation benchmark for I2V generation. Our automatic and human evaluation results demonstrate the superiority of ConsistI2V over existing methods.","url_abs":"https://arxiv.org/abs/2402.04324v2","url_pdf":"https://arxiv.org/pdf/2402.04324v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"consisti2v-enhancing-visual-consistency-for","repo_url":"https://github.com/TIGER-AI-Lab/ConsistI2V","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-to-video","task_name":"Image to Video Generation"},{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.04324","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.04324"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/TIGER-AI-Lab/ConsistI2V","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"ran_violates":2,"ran_fixture":2,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":12,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f07bf3d46feead4c","entry":"broadcat","repo":"TIGER-AI-Lab/ConsistI2V","repo_kind":"official","path":"consisti2v/models/rotary_embedding.py","file_url":"https://github.com/TIGER-AI-Lab/ConsistI2V/blob/HEAD/consisti2v/models/rotary_embedding.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f07bf3d46feead4c"}},{"code_sha256_prefix":"60fff7c3c400d7ff","entry":"default","repo":"TIGER-AI-Lab/ConsistI2V","repo_kind":"official","path":"consisti2v/models/rotary_embedding.py","file_url":"https://github.com/TIGER-AI-Lab/ConsistI2V/blob/HEAD/consisti2v/models/rotary_embedding.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"60fff7c3c400d7ff"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"TIGER-AI-Lab/ConsistI2V","repo_kind":"official","path":"consisti2v/models/rotary_embedding.py","file_url":"https://github.com/TIGER-AI-Lab/ConsistI2V/blob/HEAD/consisti2v/models/rotary_embedding.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"e285f4a541803f96","entry":"freq_mix_3d","repo":"TIGER-AI-Lab/ConsistI2V","repo_kind":"official","path":"consisti2v/utils/frameinit_utils.py","file_url":"https://github.com/TIGER-AI-Lab/ConsistI2V/blob/HEAD/consisti2v/utils/frameinit_utils.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e285f4a541803f96"}},{"code_sha256_prefix":"5753a1ae1b2b2a7a","entry":"gaussian_low_pass_filter","repo":"TIGER-AI-Lab/ConsistI2V","repo_kind":"official","path":"consisti2v/utils/frameinit_utils.py","file_url":"https://github.com/TIGER-AI-Lab/ConsistI2V/blob/HEAD/consisti2v/utils/frameinit_utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5753a1ae1b2b2a7a"}},{"code_sha256_prefix":"e6e9076eaee4d40e","entry":"get_freq_filter","repo":"TIGER-AI-Lab/ConsistI2V","repo_kind":"official","path":"consisti2v/utils/frameinit_utils.py","file_url":"https://github.com/TIGER-AI-Lab/ConsistI2V/blob/HEAD/consisti2v/utils/frameinit_utils.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e6e9076eaee4d40e"}},{"code_sha256_prefix":"5c98cd544c2fe28d","entry":"get_noise_pred_single","repo":"TIGER-AI-Lab/ConsistI2V","repo_kind":"official","path":"consisti2v/utils/util.py","file_url":"https://github.com/TIGER-AI-Lab/ConsistI2V/blob/HEAD/consisti2v/utils/util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5c98cd544c2fe28d"}},{"code_sha256_prefix":"fb032d6fa7e6196d","entry":"next_step","repo":"TIGER-AI-Lab/ConsistI2V","repo_kind":"official","path":"consisti2v/utils/util.py","file_url":"https://github.com/TIGER-AI-Lab/ConsistI2V/blob/HEAD/consisti2v/utils/util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fb032d6fa7e6196d"}},{"code_sha256_prefix":"6a8efc5f6a39c807","entry":"pan_left","repo":"TIGER-AI-Lab/ConsistI2V","repo_kind":"official","path":"consisti2v/pipelines/pipeline_conditional_animation.py","file_url":"https://github.com/TIGER-AI-Lab/ConsistI2V/blob/HEAD/consisti2v/pipelines/pipeline_conditional_animation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a8efc5f6a39c807"}},{"code_sha256_prefix":"b2e8efa1d185957b","entry":"pan_right","repo":"TIGER-AI-Lab/ConsistI2V","repo_kind":"official","path":"consisti2v/pipelines/pipeline_conditional_animation.py","file_url":"https://github.com/TIGER-AI-Lab/ConsistI2V/blob/HEAD/consisti2v/pipelines/pipeline_conditional_animation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b2e8efa1d185957b"}},{"code_sha256_prefix":"bea2d776a332f2b0","entry":"rescale_noise_cfg","repo":"TIGER-AI-Lab/ConsistI2V","repo_kind":"official","path":"consisti2v/pipelines/pipeline_autoregress_animation.py","file_url":"https://github.com/TIGER-AI-Lab/ConsistI2V/blob/HEAD/consisti2v/pipelines/pipeline_autoregress_animation.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bea2d776a332f2b0"}},{"code_sha256_prefix":"a0a4d94af120c56f","entry":"init_prompt","repo":"TIGER-AI-Lab/ConsistI2V","repo_kind":"official","path":"consisti2v/utils/util.py","file_url":"https://github.com/TIGER-AI-Lab/ConsistI2V/blob/HEAD/consisti2v/utils/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a0a4d94af120c56f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}