{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/vrope-rotary-position-embedding-for-video","title":"VRoPE: Rotary Position Embedding for Video Large Language Models","arxiv_id":"2502.11664","date":"2025-02-17","proceeding":null,"authors":["Zikang Liu","Longteng Guo","Yepeng Tang","Tongtian Yue","Junxian Cai","Kai Ma","Qingbin Liu","Xi Chen","Jing Liu"],"abstract":"Rotary Position Embedding (RoPE) has shown strong performance in text-based Large Language Models (LLMs), but extending it to video remains a challenge due to the intricate spatiotemporal structure of video frames. Existing adaptations, such as RoPE-3D, attempt to encode spatial and temporal dimensions separately but suffer from two major limitations: positional bias in attention distribution and disruptions in video-text transitions. To overcome these issues, we propose Video Rotary Position Embedding (VRoPE), a novel positional encoding method tailored for Video-LLMs. Specifically, we introduce a more balanced encoding strategy that mitigates attention biases, ensuring a more uniform distribution of spatial focus. Additionally, our approach restructures positional indices to ensure a smooth transition between video and text tokens. Extensive experiments on different models demonstrate that VRoPE consistently outperforms previous RoPE variants, achieving significant improvements in video understanding, temporal reasoning, and retrieval tasks. Code will be available at https://github.com/johncaged/VRoPE.","url_abs":"https://arxiv.org/abs/2502.11664v2","url_pdf":"https://arxiv.org/pdf/2502.11664v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"vrope-rotary-position-embedding-for-video","repo_url":"https://github.com/johncaged/vrope","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":null,"task_name":"Position"},{"task_slug":"video-understanding","task_name":"Video Understanding"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2502.11664","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.11664"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/johncaged/VRoPE","reach":null}],"summary":{"ran":2,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"31faa5358354b0c9","entry":"CustomRoPE","repo":"johncaged/VRoPE","repo_kind":"official","path":"vrope.py","file_url":"https://github.com/johncaged/VRoPE/blob/HEAD/vrope.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"31faa5358354b0c9"}},{"code_sha256_prefix":"e3dd67c1637f7270","entry":"VRoPE","repo":"johncaged/VRoPE","repo_kind":"official","path":"vrope.py","file_url":"https://github.com/johncaged/VRoPE/blob/HEAD/vrope.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e3dd67c1637f7270"}},{"code_sha256_prefix":"e438b0bb0d59b06c","entry":"DataShape","repo":"johncaged/VRoPE","repo_kind":"official","path":"vrope.py","file_url":"https://github.com/johncaged/VRoPE/blob/HEAD/vrope.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e438b0bb0d59b06c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}