{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ltx-video-realtime-video-latent-diffusion","title":"LTX-Video: Realtime Video Latent Diffusion","arxiv_id":"2501.00103","date":"2024-12-30","proceeding":null,"authors":["Yoav HaCohen","Nisan Chiprut","Benny Brazowski","Daniel Shalem","Dudu Moshe","Eitan Richardson","Eran Levin","Guy Shiran","Nir Zabari","Ori Gordon","Poriya Panet","Sapir Weissbuch","Victor Kulikov","Yaki Bitterman","Zeev Melumian","Ofir Bibi"],"abstract":"We introduce LTX-Video, a transformer-based latent diffusion model that adopts a holistic approach to video generation by seamlessly integrating the responsibilities of the Video-VAE and the denoising transformer. Unlike existing methods, which treat these components as independent, LTX-Video aims to optimize their interaction for improved efficiency and quality. At its core is a carefully designed Video-VAE that achieves a high compression ratio of 1:192, with spatiotemporal downscaling of 32 x 32 x 8 pixels per token, enabled by relocating the patchifying operation from the transformer's input to the VAE's input. Operating in this highly compressed latent space enables the transformer to efficiently perform full spatiotemporal self-attention, which is essential for generating high-resolution videos with temporal consistency. However, the high compression inherently limits the representation of fine details. To address this, our VAE decoder is tasked with both latent-to-pixel conversion and the final denoising step, producing the clean result directly in pixel space. This approach preserves the ability to generate fine details without incurring the runtime cost of a separate upsampling module. Our model supports diverse use cases, including text-to-video and image-to-video generation, with both capabilities trained simultaneously. It achieves faster-than-real-time generation, producing 5 seconds of 24 fps video at 768x512 resolution in just 2 seconds on an Nvidia H100 GPU, outperforming all existing models of similar scale. The source code and pre-trained models are publicly available, setting a new benchmark for accessible and scalable video generation.","url_abs":"https://arxiv.org/abs/2501.00103v1","url_pdf":"https://arxiv.org/pdf/2501.00103v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ltx-video-realtime-video-latent-diffusion","repo_url":"https://github.com/Lightricks/LTX-Video","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":null,"task_name":"GPU"},{"task_slug":"image-to-video","task_name":"Image to Video Generation"},{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"latent-diffusion-model","method_name":"Latent Diffusion Model"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2501.00103","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2501.00103"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Lightricks/LTX-Video","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":9},"by_repo_kind":{"official":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2cd75db71f588c1a","entry":"calculate_padding","repo":"Lightricks/LTX-Video","repo_kind":"official","path":"ltx_video/inference.py","file_url":"https://github.com/Lightricks/LTX-Video/blob/HEAD/ltx_video/inference.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2cd75db71f588c1a"}},{"code_sha256_prefix":"4a1fa9dbdc1fc119","entry":"convert_prompt_to_filename","repo":"Lightricks/LTX-Video","repo_kind":"official","path":"ltx_video/inference.py","file_url":"https://github.com/Lightricks/LTX-Video/blob/HEAD/ltx_video/inference.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4a1fa9dbdc1fc119"}},{"code_sha256_prefix":"7f8c63ce4e6a7ebc","entry":"create_video_autoencoder_config","repo":"Lightricks/LTX-Video","repo_kind":"official","path":"ltx_video/models/autoencoders/video_autoencoder.py","file_url":"https://github.com/Lightricks/LTX-Video/blob/HEAD/ltx_video/models/autoencoders/video_autoencoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7f8c63ce4e6a7ebc"}},{"code_sha256_prefix":"0b068126ba4f524b","entry":"create_video_autoencoder_demo_config","repo":"Lightricks/LTX-Video","repo_kind":"official","path":"ltx_video/models/autoencoders/causal_video_autoencoder.py","file_url":"https://github.com/Lightricks/LTX-Video/blob/HEAD/ltx_video/models/autoencoders/causal_video_autoencoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0b068126ba4f524b"}},{"code_sha256_prefix":"64056a38654e0a7e","entry":"make_linear_nd","repo":"Lightricks/LTX-Video","repo_kind":"official","path":"ltx_video/models/autoencoders/conv_nd_factory.py","file_url":"https://github.com/Lightricks/LTX-Video/blob/HEAD/ltx_video/models/autoencoders/conv_nd_factory.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"64056a38654e0a7e"}},{"code_sha256_prefix":"b66d92e3db33734e","entry":"patchify","repo":"Lightricks/LTX-Video","repo_kind":"official","path":"ltx_video/models/autoencoders/causal_video_autoencoder.py","file_url":"https://github.com/Lightricks/LTX-Video/blob/HEAD/ltx_video/models/autoencoders/causal_video_autoencoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b66d92e3db33734e"}},{"code_sha256_prefix":"79dc833d8e0d8a46","entry":"patchify","repo":"Lightricks/LTX-Video","repo_kind":"official","path":"ltx_video/models/autoencoders/video_autoencoder.py","file_url":"https://github.com/Lightricks/LTX-Video/blob/HEAD/ltx_video/models/autoencoders/video_autoencoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"79dc833d8e0d8a46"}},{"code_sha256_prefix":"0dcf9131a653903b","entry":"unpatchify","repo":"Lightricks/LTX-Video","repo_kind":"official","path":"ltx_video/models/autoencoders/causal_video_autoencoder.py","file_url":"https://github.com/Lightricks/LTX-Video/blob/HEAD/ltx_video/models/autoencoders/causal_video_autoencoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0dcf9131a653903b"}},{"code_sha256_prefix":"48aaf82ac0aedbb6","entry":"unpatchify","repo":"Lightricks/LTX-Video","repo_kind":"official","path":"ltx_video/models/autoencoders/video_autoencoder.py","file_url":"https://github.com/Lightricks/LTX-Video/blob/HEAD/ltx_video/models/autoencoders/video_autoencoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"48aaf82ac0aedbb6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}