{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/kwai-keye-vl-technical-report","title":"Kwai Keye-VL Technical Report","arxiv_id":"2507.01949","date":"2025-07-02","proceeding":null,"authors":["Kwai Keye Team","Biao Yang","Bin Wen","Changyi Liu","Chenglong Chu","Chengru Song","Chongling Rao","Chuan Yi","Da Li","Dunju Zang","Fan Yang","Guorui Zhou","Hao Peng","Haojie Ding","Jiaming Huang","Jiangxia Cao","Jiankang Chen","Jingyun Hua","Jin Ouyang","Kaibing Chen","Kaiyu Jiang","Kaiyu Tang","Kun Gai","ShengNan Zhang","Siyang Mao","Sui Huang","Tianke Zhang","Tingting Gao","Wei Chen","Wei Yuan","Xiangyu Wu","Xiao Hu","Xingyu Lu","Yang Zhou","Yi-Fan Zhang","Yiping Yang","Yulong Chen","Zhenhua Wu","Zhenyu Li","Zhixin Ling","Ziming Li","Dehua Ma","Di Xu","Haixuan Gao","Hang Li","Jiawei Guo","Jing Wang","Lejian Ren","Muhao Wei","Qianqian Wang","Qigen Hu","Shiyao Wang","Tao Yu","Xinchen Luo","Yan Li","Yiming Liang","Yuhang Hu","Zeyi Lu","Zhuoran Yang","Zixing Zhang"],"abstract":"While Multimodal Large Language Models (MLLMs) demonstrate remarkable capabilities on static images, they often fall short in comprehending dynamic, information-dense short-form videos, a dominant medium in today's digital landscape. To bridge this gap, we introduce \\textbf{Kwai Keye-VL}, an 8-billion-parameter multimodal foundation model engineered for leading-edge performance in short-video understanding while maintaining robust general-purpose vision-language abilities. The development of Keye-VL rests on two core pillars: a massive, high-quality dataset exceeding 600 billion tokens with a strong emphasis on video, and an innovative training recipe. This recipe features a four-stage pre-training process for solid vision-language alignment, followed by a meticulous two-phase post-training process. The first post-training stage enhances foundational capabilities like instruction following, while the second phase focuses on stimulating advanced reasoning. In this second phase, a key innovation is our five-mode ``cold-start'' data mixture, which includes ``thinking'', ``non-thinking'', ``auto-think'', ``think with image'', and high-quality video data. This mixture teaches the model to decide when and how to reason. Subsequent reinforcement learning (RL) and alignment steps further enhance these reasoning capabilities and correct abnormal model behaviors, such as repetitive outputs. To validate our approach, we conduct extensive evaluations, showing that Keye-VL achieves state-of-the-art results on public video benchmarks and remains highly competitive on general image-based tasks (Figure 1). Furthermore, we develop and release the \\textbf{KC-MMBench}, a new benchmark tailored for real-world short-video scenarios, where Keye-VL shows a significant advantage.","url_abs":"https://arxiv.org/abs/2507.01949v1","url_pdf":"https://arxiv.org/pdf/2507.01949v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"kwai-keye-vl-technical-report","repo_url":"https://github.com/kwai-keye/keye","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"instruction-following","task_name":"Instruction Following"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"video-understanding","task_name":"Video Understanding"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2507.01949","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2507.01949"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kwai-keye/keye","reach":{"status":"ok"}}],"summary":{"ran_honours":3,"ran_draft_wrong":1,"unverified":9},"by_repo_kind":{"official":{"samples":13,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":13,"samples":[{"code_sha256_prefix":"6e45201fa27cb24a","entry":"ceil_by_factor","repo":"kwai-keye/keye","repo_kind":"official","path":"keye-vl-utils/src/keye_vl_utils/vision_process.py","file_url":"https://github.com/kwai-keye/keye/blob/HEAD/keye-vl-utils/src/keye_vl_utils/vision_process.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6e45201fa27cb24a"}},{"code_sha256_prefix":"8155263d7ff19bb3","entry":"floor_by_factor","repo":"kwai-keye/keye","repo_kind":"official","path":"keye-vl-utils/src/keye_vl_utils/vision_process.py","file_url":"https://github.com/kwai-keye/keye/blob/HEAD/keye-vl-utils/src/keye_vl_utils/vision_process.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8155263d7ff19bb3"}},{"code_sha256_prefix":"e03d53ba9d4f9ae5","entry":"rotate_half","repo":"kwai-keye/keye","repo_kind":"official","path":"keye-vl-8b-preview/modeling_keye.py","file_url":"https://github.com/kwai-keye/keye/blob/HEAD/keye-vl-8b-preview/modeling_keye.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e03d53ba9d4f9ae5"}},{"code_sha256_prefix":"e252767324188623","entry":"round_by_factor","repo":"kwai-keye/keye","repo_kind":"official","path":"keye-vl-utils/src/keye_vl_utils/vision_process.py","file_url":"https://github.com/kwai-keye/keye/blob/HEAD/keye-vl-utils/src/keye_vl_utils/vision_process.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e252767324188623"}},{"code_sha256_prefix":"1701d522f51b84e5","entry":"adjust_size","repo":"kwai-keye/keye","repo_kind":"official","path":"keye-vl-8b-preview/image_processing_keye.py","file_url":"https://github.com/kwai-keye/keye/blob/HEAD/keye-vl-8b-preview/image_processing_keye.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1701d522f51b84e5"}},{"code_sha256_prefix":"c85c6c625d62983a","entry":"apply_rotary_pos_emb_flashatt","repo":"kwai-keye/keye","repo_kind":"official","path":"keye-vl-8b-preview/modeling_keye.py","file_url":"https://github.com/kwai-keye/keye/blob/HEAD/keye-vl-8b-preview/modeling_keye.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c85c6c625d62983a"}},{"code_sha256_prefix":"c96f86fc774ee85d","entry":"chat_mt","repo":"kwai-keye/keye","repo_kind":"official","path":"evaluation/KC-MMBench/vlmeval/inference_mt.py","file_url":"https://github.com/kwai-keye/keye/blob/HEAD/evaluation/KC-MMBench/vlmeval/inference_mt.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c96f86fc774ee85d"}},{"code_sha256_prefix":"ca192302458c9c1f","entry":"eager_attention_forward","repo":"kwai-keye/keye","repo_kind":"official","path":"keye-vl-8b-preview/modeling_keye.py","file_url":"https://github.com/kwai-keye/keye/blob/HEAD/keye-vl-8b-preview/modeling_keye.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ca192302458c9c1f"}},{"code_sha256_prefix":"a5f4263865d5afb8","entry":"ensure_image_url","repo":"kwai-keye/keye","repo_kind":"official","path":"evaluation/KC-MMBench/vlmeval/vlm/keye_vlm/model.py","file_url":"https://github.com/kwai-keye/keye/blob/HEAD/evaluation/KC-MMBench/vlmeval/vlm/keye_vlm/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a5f4263865d5afb8"}},{"code_sha256_prefix":"eabaa9c5d46a1a4f","entry":"ensure_video_url","repo":"kwai-keye/keye","repo_kind":"official","path":"evaluation/KC-MMBench/vlmeval/vlm/keye_vlm/model.py","file_url":"https://github.com/kwai-keye/keye/blob/HEAD/evaluation/KC-MMBench/vlmeval/vlm/keye_vlm/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eabaa9c5d46a1a4f"}},{"code_sha256_prefix":"6517db7a566fb96d","entry":"get_gpu_num","repo":"kwai-keye/keye","repo_kind":"official","path":"evaluation/KC-MMBench/vlmeval/api/hf_chat_model.py","file_url":"https://github.com/kwai-keye/keye/blob/HEAD/evaluation/KC-MMBench/vlmeval/api/hf_chat_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6517db7a566fb96d"}},{"code_sha256_prefix":"0132226f8e843f1b","entry":"make_batched_images","repo":"kwai-keye/keye","repo_kind":"official","path":"keye-vl-8b-preview/image_processing_keye.py","file_url":"https://github.com/kwai-keye/keye/blob/HEAD/keye-vl-8b-preview/image_processing_keye.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0132226f8e843f1b"}},{"code_sha256_prefix":"5639465a53a0ec48","entry":"make_batched_videos","repo":"kwai-keye/keye","repo_kind":"official","path":"keye-vl-8b-preview/image_processing_keye.py","file_url":"https://github.com/kwai-keye/keye/blob/HEAD/keye-vl-8b-preview/image_processing_keye.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5639465a53a0ec48"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}