{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scaling-rl-to-long-videos","title":"Scaling RL to Long Videos","arxiv_id":"2507.07966","date":"2025-07-10","proceeding":null,"authors":["Yukang Chen","Wei Huang","Baifeng Shi","Qinghao Hu","Hanrong Ye","Ligeng Zhu","Zhijian Liu","Pavlo Molchanov","Jan Kautz","Xiaojuan Qi","Sifei Liu","Hongxu Yin","Yao Lu","Song Han"],"abstract":"We introduce a full-stack framework that scales up reasoning in vision-language models (VLMs) to long videos, leveraging reinforcement learning. We address the unique challenges of long video reasoning by integrating three critical components: (1) a large-scale dataset, LongVideo-Reason, comprising 52K long video QA pairs with high-quality reasoning annotations across diverse domains such as sports, games, and vlogs; (2) a two-stage training pipeline that extends VLMs with chain-of-thought supervised fine-tuning (CoT-SFT) and reinforcement learning (RL); and (3) a training infrastructure for long video RL, named Multi-modal Reinforcement Sequence Parallelism (MR-SP), which incorporates sequence parallelism and a vLLM-based engine tailored for long video, using cached video embeddings for efficient rollout and prefilling. In experiments, LongVILA-R1-7B achieves strong performance on long video QA benchmarks such as VideoMME. It also outperforms Video-R1-7B and even matches Gemini-1.5-Pro across temporal reasoning, goal and purpose reasoning, spatial reasoning, and plot reasoning on our LongVideo-Reason-eval benchmark. Notably, our MR-SP system achieves up to 2.1x speedup on long video RL training. LongVILA-R1 demonstrates consistent performance gains as the number of input video frames scales. LongVILA-R1 marks a firm step towards long video reasoning in VLMs. In addition, we release our training system for public availability that supports RL training on various modalities (video, text, and audio), various models (VILA and Qwen series), and even image and video generation models. On a single A100 node (8 GPUs), it supports RL training on hour-long videos (e.g., 3,600 frames / around 256k tokens).","url_abs":"https://arxiv.org/abs/2507.07966v1","url_pdf":"https://arxiv.org/pdf/2507.07966v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scaling-rl-to-long-videos","repo_url":"https://github.com/hiyouga/easyr1","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"spatial-reasoning","task_name":"Spatial Reasoning"},{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2507.07966","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2507.07966"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/NVlabs/Long-RL","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hiyouga/easyr1","reach":null}],"summary":{"ran_violates":1,"ran":3,"ran_draft_wrong":3,"ran_fixture":1,"unverified":16},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1},"found_in_text":{"samples":18,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8dd6b65f72c07727","entry":"check_dot_in_model_path","repo":"nvlabs/long-rl","repo_kind":"found_in_text","path":"verl/utils/vila_remote_code/modeling_vila.py","file_url":"https://github.com/nvlabs/long-rl/blob/HEAD/verl/utils/vila_remote_code/modeling_vila.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":2,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8dd6b65f72c07727"}},{"code_sha256_prefix":"9ab20c0df51491c6","entry":"SeqAllToAll","repo":"hiyouga/easyr1","repo_kind":"official","path":"verl/utils/ulysses.py","file_url":"https://github.com/hiyouga/easyr1/blob/HEAD/verl/utils/ulysses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9ab20c0df51491c6"}},{"code_sha256_prefix":"bc1859396d0efde4","entry":"_unpad_tensor","repo":"hiyouga/easyr1","repo_kind":"official","path":"verl/utils/ulysses.py","file_url":"https://github.com/hiyouga/easyr1/blob/HEAD/verl/utils/ulysses.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bc1859396d0efde4"}},{"code_sha256_prefix":"c370c3e6e37ff52d","entry":"gather_seq_scatter_heads","repo":"hiyouga/easyr1","repo_kind":"official","path":"verl/utils/ulysses.py","file_url":"https://github.com/hiyouga/easyr1/blob/HEAD/verl/utils/ulysses.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c370c3e6e37ff52d"}},{"code_sha256_prefix":"5bc06ef1058a26db","entry":"get_ulysses_sequence_parallel_world_size","repo":"hiyouga/easyr1","repo_kind":"official","path":"verl/utils/ulysses.py","file_url":"https://github.com/hiyouga/easyr1/blob/HEAD/verl/utils/ulysses.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OPAQUE_PARAMS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5bc06ef1058a26db"}},{"code_sha256_prefix":"554c742ab091b991","entry":"get_vila_version","repo":"nvlabs/long-rl","repo_kind":"found_in_text","path":"verl/utils/vila_remote_code/modeling_vila.py","file_url":"https://github.com/nvlabs/long-rl/blob/HEAD/verl/utils/vila_remote_code/modeling_vila.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"554c742ab091b991"}},{"code_sha256_prefix":"4049cf08f994464e","entry":"prepare_fa2_from_position_ids","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"verl/models/transformers/flash_attention_utils.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/verl/models/transformers/flash_attention_utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4049cf08f994464e"}},{"code_sha256_prefix":"a91c18341292d50e","entry":"unpad_dataproto","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"verl/protocol.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/verl/protocol.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a91c18341292d50e"}},{"code_sha256_prefix":"50fceeb8e224ae4b","entry":"all_to_all_tensor","repo":"hiyouga/easyr1","repo_kind":"official","path":"verl/utils/ulysses.py","file_url":"https://github.com/hiyouga/easyr1/blob/HEAD/verl/utils/ulysses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"50fceeb8e224ae4b"}},{"code_sha256_prefix":"defc3c221f010765","entry":"collect_video_files","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"longvideo-reason/step3_merge_captions.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/longvideo-reason/step3_merge_captions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"defc3c221f010765"}},{"code_sha256_prefix":"b0bfe8a6b7ced02a","entry":"compute_gae_advantage_return","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"verl/trainer/core_algos.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/verl/trainer/core_algos.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b0bfe8a6b7ced02a"}},{"code_sha256_prefix":"758c7f607efffb52","entry":"compute_grpo_outcome_advantage","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"verl/trainer/core_algos.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/verl/trainer/core_algos.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"758c7f607efffb52"}},{"code_sha256_prefix":"2058875a75c3ff14","entry":"flash_attention_forward","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"verl/models/transformers/flash_attention_utils.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/verl/models/transformers/flash_attention_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2058875a75c3ff14"}},{"code_sha256_prefix":"6e021f58ff92252d","entry":"generate_captions","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"longvideo-reason/step4_gen_reasoning_data.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/longvideo-reason/step4_gen_reasoning_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6e021f58ff92252d"}},{"code_sha256_prefix":"c02118fc07296220","entry":"generate_gpt","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"longvideo-reason/step6_reformat_reasoning_data.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/longvideo-reason/step6_reformat_reasoning_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c02118fc07296220"}},{"code_sha256_prefix":"452188d5e8665c1e","entry":"get_chunked_index","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"verl/models/transformers/qwen2_5_omni.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/verl/models/transformers/qwen2_5_omni.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"452188d5e8665c1e"}},{"code_sha256_prefix":"7633fd09bcf8972a","entry":"get_kl_controller","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"verl/trainer/core_algos.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/verl/trainer/core_algos.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7633fd09bcf8972a"}},{"code_sha256_prefix":"2b24afde621bf5ff","entry":"get_llm_pos_ids_for_vision","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"verl/models/transformers/qwen2_5_omni.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/verl/models/transformers/qwen2_5_omni.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2b24afde621bf5ff"}},{"code_sha256_prefix":"867399f13a93e09a","entry":"get_rope_index_omni","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"verl/models/transformers/qwen2_5_omni.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/verl/models/transformers/qwen2_5_omni.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"867399f13a93e09a"}},{"code_sha256_prefix":"c3db68c3c1890a1b","entry":"get_ulysses_sequence_parallel_group","repo":"hiyouga/easyr1","repo_kind":"official","path":"verl/utils/ulysses.py","file_url":"https://github.com/hiyouga/easyr1/blob/HEAD/verl/utils/ulysses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c3db68c3c1890a1b"}},{"code_sha256_prefix":"34e4399c3e46211a","entry":"merge_captions","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"longvideo-reason/step3_merge_captions.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/longvideo-reason/step3_merge_captions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"34e4399c3e46211a"}},{"code_sha256_prefix":"c7a1430e0e702826","entry":"pad_dataproto_to_divisor","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"verl/protocol.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/verl/protocol.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c7a1430e0e702826"}},{"code_sha256_prefix":"d9f1e79118788b85","entry":"parse_reasoning","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"longvideo-reason/step5_parse_reasoning_data.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/longvideo-reason/step5_parse_reasoning_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d9f1e79118788b85"}},{"code_sha256_prefix":"ad658a1d6fe829ba","entry":"union_tensor_dict","repo":"NVlabs/Long-RL","repo_kind":"found_in_text","path":"verl/protocol.py","file_url":"https://github.com/NVlabs/Long-RL/blob/HEAD/verl/protocol.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ad658a1d6fe829ba"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}