{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/video-diffusion-alignment-via-reward","title":"Video Diffusion Alignment via Reward Gradients","arxiv_id":"2407.08737","date":"2024-07-11","proceeding":null,"authors":["Mihir Prabhudesai","Russell Mendonca","Zheyang Qin","Katerina Fragkiadaki","Deepak Pathak"],"abstract":"We have made significant progress towards building foundational video diffusion models. As these models are trained using large-scale unsupervised data, it has become crucial to adapt these models to specific downstream tasks. Adapting these models via supervised fine-tuning requires collecting target datasets of videos, which is challenging and tedious. In this work, we utilize pre-trained reward models that are learned via preferences on top of powerful vision discriminative models to adapt video diffusion models. These models contain dense gradient information with respect to generated RGB pixels, which is critical to efficient learning in complex search spaces, such as videos. We show that backpropagating gradients from these reward models to a video diffusion model can allow for compute and sample efficient alignment of the video diffusion model. We show results across a variety of reward models and video diffusion models, demonstrating that our approach can learn much more efficiently in terms of reward queries and computation than prior gradient-free approaches. Our code, model weights,and more visualization are available at https://vader-vid.github.io.","url_abs":"https://arxiv.org/abs/2407.08737v1","url_pdf":"https://arxiv.org/pdf/2407.08737v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"video-diffusion-alignment-via-reward","repo_url":"https://github.com/mihirp1998/vader","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.08737","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.08737"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mihirp1998/vader","reach":null}],"summary":{"ran_draft_wrong":3,"ran_violates":1},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"ee27b545d70072ef","entry":"create_output_folders","repo":"mihirp1998/vader","repo_kind":"official","path":"VADER-VideoCrafter/scripts/main/train_t2v_lora.py","file_url":"https://github.com/mihirp1998/vader/blob/HEAD/VADER-VideoCrafter/scripts/main/train_t2v_lora.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ee27b545d70072ef"}},{"code_sha256_prefix":"aed26442868059ab","entry":"create_output_folders","repo":"mihirp1998/vader","repo_kind":"official","path":"VADER-ModelScope/train_t2v_lora.py","file_url":"https://github.com/mihirp1998/vader/blob/HEAD/VADER-ModelScope/train_t2v_lora.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aed26442868059ab"}},{"code_sha256_prefix":"4481bf18d0cda4a2","entry":"is_attn","repo":"mihirp1998/vader","repo_kind":"official","path":"VADER-ModelScope/train_t2v_lora.py","file_url":"https://github.com/mihirp1998/vader/blob/HEAD/VADER-ModelScope/train_t2v_lora.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4481bf18d0cda4a2"}},{"code_sha256_prefix":"12ac91dbd8875081","entry":"str2bool","repo":"mihirp1998/vader","repo_kind":"official","path":"VADER-VideoCrafter/scripts/main/train_t2v_lora.py","file_url":"https://github.com/mihirp1998/vader/blob/HEAD/VADER-VideoCrafter/scripts/main/train_t2v_lora.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"12ac91dbd8875081"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}