{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ca2-vdm-efficient-autoregressive-video","title":"Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharing","arxiv_id":"2411.16375","date":"2024-11-25","proceeding":null,"authors":["Kaifeng Gao","Jiaxin Shi","Hanwang Zhang","Chunping Wang","Jun Xiao","Long Chen"],"abstract":"With the advance of diffusion models, today's video generation has achieved impressive quality. To extend the generation length and facilitate real-world applications, a majority of video diffusion models (VDMs) generate videos in an autoregressive manner, i.e., generating subsequent clips conditioned on the last frame(s) of the previous clip. However, existing autoregressive VDMs are highly inefficient and redundant: The model must re-compute all the conditional frames that are overlapped between adjacent clips. This issue is exacerbated when the conditional frames are extended autoregressively to provide the model with long-term context. In such cases, the computational demands increase significantly (i.e., with a quadratic complexity w.r.t. the autoregression step). In this paper, we propose Ca2-VDM, an efficient autoregressive VDM with Causal generation and Cache sharing. For causal generation, it introduces unidirectional feature computation, which ensures that the cache of conditional frames can be precomputed in previous autoregression steps and reused in every subsequent step, eliminating redundant computations. For cache sharing, it shares the cache across all denoising steps to avoid the huge cache storage cost. Extensive experiments demonstrated that our Ca2-VDM achieves state-of-the-art quantitative and qualitative video generation results and significantly improves the generation speed. Code is available: https://github.com/Dawn-LX/CausalCache-VDM","url_abs":"https://arxiv.org/abs/2411.16375v2","url_pdf":"https://arxiv.org/pdf/2411.16375v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ca2-vdm-efficient-autoregressive-video","repo_url":"https://github.com/dawn-lx/causalcache-vdm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"ca2-vdm-efficient-autoregressive-video","repo_url":"https://github.com/dawn-lx/causal-videogen","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2411.16375","atlas_url":"https://app.syntology.ai/?focus=2411.16375","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.16375"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/songweige/TATS","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/universome/stylegan-v","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dawn-lx/causalcache-vdm","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dawn-lx/causal-videogen","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":7,"ran_fixture":2,"unverified":12},"by_repo_kind":{"listed":{"samples":6,"ran":0,"repositories":1},"found_in_text":{"samples":15,"ran":9,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"c36e0664ff375f5b","entry":"Block","repo":"songweige/TATS","repo_kind":"found_in_text","path":"tats/tats_transformer.py","file_url":"https://github.com/songweige/TATS/blob/HEAD/tats/tats_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c36e0664ff375f5b"}},{"code_sha256_prefix":"0dcc3d2b5989bb48","entry":"CausalSelfAttention","repo":"songweige/TATS","repo_kind":"found_in_text","path":"tats/tats_transformer.py","file_url":"https://github.com/songweige/TATS/blob/HEAD/tats/tats_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0dcc3d2b5989bb48"}},{"code_sha256_prefix":"20e50a0db3aadd39","entry":"GPT","repo":"songweige/TATS","repo_kind":"found_in_text","path":"tats/tats_transformer.py","file_url":"https://github.com/songweige/TATS/blob/HEAD/tats/tats_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"20e50a0db3aadd39"}},{"code_sha256_prefix":"cd407b6b077bc83e","entry":"GPTConfig","repo":"songweige/TATS","repo_kind":"found_in_text","path":"tats/tats_transformer.py","file_url":"https://github.com/songweige/TATS/blob/HEAD/tats/tats_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd407b6b077bc83e"}},{"code_sha256_prefix":"f020bb1b02990123","entry":"Identity","repo":"songweige/TATS","repo_kind":"found_in_text","path":"tats/tats_transformer.py","file_url":"https://github.com/songweige/TATS/blob/HEAD/tats/tats_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f020bb1b02990123"}},{"code_sha256_prefix":"b9f40f41b93875c6","entry":"Labelator","repo":"songweige/TATS","repo_kind":"found_in_text","path":"tats/tats_transformer.py","file_url":"https://github.com/songweige/TATS/blob/HEAD/tats/tats_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b9f40f41b93875c6"}},{"code_sha256_prefix":"6e1c0c9a43690fdb","entry":"SOSProvider","repo":"songweige/TATS","repo_kind":"found_in_text","path":"tats/tats_transformer.py","file_url":"https://github.com/songweige/TATS/blob/HEAD/tats/tats_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6e1c0c9a43690fdb"}},{"code_sha256_prefix":"085a09050f4f41dc","entry":"accuracy","repo":"songweige/TATS","repo_kind":"found_in_text","path":"tats/tats_transformer.py","file_url":"https://github.com/songweige/TATS/blob/HEAD/tats/tats_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"085a09050f4f41dc"}},{"code_sha256_prefix":"791774067a83bf99","entry":"shift_dim","repo":"songweige/TATS","repo_kind":"found_in_text","path":"tats/tats_transformer.py","file_url":"https://github.com/songweige/TATS/blob/HEAD/tats/tats_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"791774067a83bf99"}},{"code_sha256_prefix":"af840b748b65873d","entry":"Net2NetTransformer","repo":"songweige/TATS","repo_kind":"found_in_text","path":"tats/tats_transformer.py","file_url":"https://github.com/songweige/TATS/blob/HEAD/tats/tats_transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"af840b748b65873d"}},{"code_sha256_prefix":"b51f59051d16f7df","entry":"all_to_all","repo":"dawn-lx/causal-videogen","repo_kind":"listed","path":"opensora/acceleration/communications.py","file_url":"https://github.com/dawn-lx/causal-videogen/blob/HEAD/opensora/acceleration/communications.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b51f59051d16f7df"}},{"code_sha256_prefix":"20336f4981288332","entry":"auto_grad_checkpoint","repo":"dawn-lx/causal-videogen","repo_kind":"listed","path":"opensora/acceleration/checkpoint.py","file_url":"https://github.com/dawn-lx/causal-videogen/blob/HEAD/opensora/acceleration/checkpoint.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"20336f4981288332"}},{"code_sha256_prefix":"496f3cd2af0df2e1","entry":"build_module","repo":"dawn-lx/causal-videogen","repo_kind":"listed","path":"opensora/registry.py","file_url":"https://github.com/dawn-lx/causal-videogen/blob/HEAD/opensora/registry.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"496f3cd2af0df2e1"}},{"code_sha256_prefix":"2dc4f95b756cb2b3","entry":"cfg_to_args_str","repo":"universome/stylegan-v","repo_kind":"found_in_text","path":"src/infra/utils.py","file_url":"https://github.com/universome/stylegan-v/blob/HEAD/src/infra/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2dc4f95b756cb2b3"}},{"code_sha256_prefix":"0bf504441d3b1c0e","entry":"compute_fvd","repo":"universome/stylegan-v","repo_kind":"found_in_text","path":"src/metrics/frechet_video_distance.py","file_url":"https://github.com/universome/stylegan-v/blob/HEAD/src/metrics/frechet_video_distance.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0bf504441d3b1c0e"}},{"code_sha256_prefix":"2c35749b6eaa5bfe","entry":"construct_experiments_args","repo":"universome/stylegan-v","repo_kind":"found_in_text","path":"src/infra/slurm_batch_launch.py","file_url":"https://github.com/universome/stylegan-v/blob/HEAD/src/infra/slurm_batch_launch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2c35749b6eaa5bfe"}},{"code_sha256_prefix":"97c082db7efd0240","entry":"gather_forward_split_backward","repo":"dawn-lx/causal-videogen","repo_kind":"listed","path":"opensora/acceleration/communications.py","file_url":"https://github.com/dawn-lx/causal-videogen/blob/HEAD/opensora/acceleration/communications.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"97c082db7efd0240"}},{"code_sha256_prefix":"f29fb27caa548814","entry":"num_gpus_to_mem","repo":"universome/stylegan-v","repo_kind":"found_in_text","path":"src/infra/utils.py","file_url":"https://github.com/universome/stylegan-v/blob/HEAD/src/infra/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f29fb27caa548814"}},{"code_sha256_prefix":"5cbd445824659be5","entry":"partial_causal_flash_attn","repo":"dawn-lx/causal-videogen","repo_kind":"listed","path":"opensora/models/causal_stdit2/attention.py","file_url":"https://github.com/dawn-lx/causal-videogen/blob/HEAD/opensora/models/causal_stdit2/attention.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5cbd445824659be5"}},{"code_sha256_prefix":"a771d32bab25e50b","entry":"split_forward_gather_backward","repo":"dawn-lx/causal-videogen","repo_kind":"listed","path":"opensora/acceleration/communications.py","file_url":"https://github.com/dawn-lx/causal-videogen/blob/HEAD/opensora/acceleration/communications.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a771d32bab25e50b"}},{"code_sha256_prefix":"90b1bec775ebe652","entry":"to_dict","repo":"universome/stylegan-v","repo_kind":"found_in_text","path":"src/infra/slurm_batch_launch.py","file_url":"https://github.com/universome/stylegan-v/blob/HEAD/src/infra/slurm_batch_launch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"90b1bec775ebe652"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}