{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/flexible-diffusion-modeling-of-long-videos","title":"Flexible Diffusion Modeling of Long Videos","arxiv_id":"2205.11495","date":"2022-05-23","proceeding":null,"authors":["William Harvey","Saeid Naderiparizi","Vaden Masrani","Christian Weilbach","Frank Wood"],"abstract":"We present a framework for video modeling based on denoising diffusion probabilistic models that produces long-duration video completions in a variety of realistic environments. We introduce a generative model that can at test-time sample any arbitrary subset of video frames conditioned on any other subset and present an architecture adapted for this purpose. Doing so allows us to efficiently compare and optimize a variety of schedules for the order in which frames in a long video are sampled and use selective sparse and long-range conditioning on previously sampled frames. We demonstrate improved video modeling over prior work on a number of datasets and sample temporally coherent videos over 25 minutes in length. We additionally release a new video modeling dataset and semantically meaningful metrics based on videos generated in the CARLA autonomous driving simulator.","url_abs":"https://arxiv.org/abs/2205.11495v3","url_pdf":"https://arxiv.org/pdf/2205.11495v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"flexible-diffusion-modeling-of-long-videos","repo_url":"https://github.com/plai-group/flexible-video-diffusion-modeling","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"},{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"rain-removal","task_name":"Rain Removal"}],"methods":[{"method_slug":"carla","method_name":"CARLA"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"entropy-regularization","method_name":"Entropy Regularization"},{"method_slug":"ppo","method_name":"PPO"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/rain-removal-on-nighrain","task":"Rain Removal","dataset":"Nightrain","model":"FDM","rank_in_archive_order":3,"of":4,"metrics":{"PSNR":"23.49"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2205.11495","atlas_url":"https://app.syntology.ai/?focus=2205.11495","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}