{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deepseek-vl2-mixture-of-experts-vision","title":"DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding","arxiv_id":"2412.10302","date":"2024-12-13","proceeding":null,"authors":["Zhiyu Wu","Xiaokang Chen","Zizheng Pan","Xingchao Liu","Wen Liu","Damai Dai","Huazuo Gao","Yiyang Ma","Chengyue Wu","Bingxuan Wang","Zhenda Xie","Yu Wu","Kai Hu","Jiawei Wang","Yaofeng Sun","Yukun Li","Yishi Piao","Kang Guan","Aixin Liu","Xin Xie","Yuxiang You","Kai Dong","Xingkai Yu","Haowei Zhang","Liang Zhao","Yisong Wang","Chong Ruan"],"abstract":"We present DeepSeek-VL2, an advanced series of large Mixture-of-Experts (MoE) Vision-Language Models that significantly improves upon its predecessor, DeepSeek-VL, through two key major upgrades. For the vision component, we incorporate a dynamic tiling vision encoding strategy designed for processing high-resolution images with different aspect ratios. For the language component, we leverage DeepSeekMoE models with the Multi-head Latent Attention mechanism, which compresses Key-Value cache into latent vectors, to enable efficient inference and high throughput. Trained on an improved vision-language dataset, DeepSeek-VL2 demonstrates superior capabilities across various tasks, including but not limited to visual question answering, optical character recognition, document/table/chart understanding, and visual grounding. Our model series is composed of three variants: DeepSeek-VL2-Tiny, DeepSeek-VL2-Small and DeepSeek-VL2, with 1.0B, 2.8B and 4.5B activated parameters respectively. DeepSeek-VL2 achieves competitive or state-of-the-art performance with similar or fewer activated parameters compared to existing open-source dense and MoE-based models. Codes and pre-trained models are publicly accessible at https://github.com/deepseek-ai/DeepSeek-VL2.","url_abs":"https://arxiv.org/abs/2412.10302v1","url_pdf":"https://arxiv.org/pdf/2412.10302v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deepseek-vl2-mixture-of-experts-vision","repo_url":"https://github.com/deepseek-ai/deepseek-vl2","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"chart-understanding","task_name":"Chart Understanding"},{"task_slug":"mixture-of-experts","task_name":"Mixture-of-Experts"},{"task_slug":"optical-character-recognition","task_name":"Optical Character Recognition"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"referring-expression-comprehension","task_name":"Referring Expression Comprehension"},{"task_slug":"visual-grounding","task_name":"Visual Grounding"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2412.10302","atlas_url":"https://app.syntology.ai/?focus=2412.10302","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.10302"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/deepseek-ai/deepseek-vl2","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran":3,"unverified":9},"by_repo_kind":{"official":{"samples":13,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"96f3e857e91a8cb8","entry":"load_json","repo":"deepseek-ai/deepseek-vl2","repo_kind":"official","path":"deepseek_vl2/utils/io.py","file_url":"https://github.com/deepseek-ai/deepseek-vl2/blob/HEAD/deepseek_vl2/utils/io.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"96f3e857e91a8cb8"}},{"code_sha256_prefix":"de3c1ddb7a5bbb47","entry":"yarn_find_correction_dim","repo":"deepseek-ai/deepseek-vl2","repo_kind":"official","path":"deepseek_vl2/models/modeling_deepseek.py","file_url":"https://github.com/deepseek-ai/deepseek-vl2/blob/HEAD/deepseek_vl2/models/modeling_deepseek.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de3c1ddb7a5bbb47"}},{"code_sha256_prefix":"ed5f1f4e8c0439b2","entry":"yarn_find_correction_range","repo":"deepseek-ai/deepseek-vl2","repo_kind":"official","path":"deepseek_vl2/models/modeling_deepseek.py","file_url":"https://github.com/deepseek-ai/deepseek-vl2/blob/HEAD/deepseek_vl2/models/modeling_deepseek.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ed5f1f4e8c0439b2"}},{"code_sha256_prefix":"8dfdb3a53834d90f","entry":"yarn_get_mscale","repo":"deepseek-ai/deepseek-vl2","repo_kind":"official","path":"deepseek_vl2/models/modeling_deepseek.py","file_url":"https://github.com/deepseek-ai/deepseek-vl2/blob/HEAD/deepseek_vl2/models/modeling_deepseek.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8dfdb3a53834d90f"}},{"code_sha256_prefix":"ba86a364bce26e9b","entry":"compact_text_chunks","repo":"deepseek-ai/deepseek-vl2","repo_kind":"official","path":"deepseek_vl2/serve/app_modules/overwrites.py","file_url":"https://github.com/deepseek-ai/deepseek-vl2/blob/HEAD/deepseek_vl2/serve/app_modules/overwrites.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ba86a364bce26e9b"}},{"code_sha256_prefix":"b675d2f1fa15bd63","entry":"create_siglip_vit","repo":"deepseek-ai/deepseek-vl2","repo_kind":"official","path":"deepseek_vl2/models/siglip_vit.py","file_url":"https://github.com/deepseek-ai/deepseek-vl2/blob/HEAD/deepseek_vl2/models/siglip_vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b675d2f1fa15bd63"}},{"code_sha256_prefix":"75628f3739d6fcd3","entry":"format_output","repo":"deepseek-ai/deepseek-vl2","repo_kind":"official","path":"deepseek_vl2/serve/app_modules/utils.py","file_url":"https://github.com/deepseek-ai/deepseek-vl2/blob/HEAD/deepseek_vl2/serve/app_modules/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"75628f3739d6fcd3"}},{"code_sha256_prefix":"132894d1028abd5a","entry":"get_conv_template","repo":"deepseek-ai/deepseek-vl2","repo_kind":"official","path":"deepseek_vl2/models/conversation.py","file_url":"https://github.com/deepseek-ai/deepseek-vl2/blob/HEAD/deepseek_vl2/models/conversation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"132894d1028abd5a"}},{"code_sha256_prefix":"56ce205e9245c8c2","entry":"load_pil_images","repo":"deepseek-ai/deepseek-vl2","repo_kind":"official","path":"deepseek_vl2/utils/io.py","file_url":"https://github.com/deepseek-ai/deepseek-vl2/blob/HEAD/deepseek_vl2/utils/io.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"56ce205e9245c8c2"}},{"code_sha256_prefix":"8a1727ee6e04ec45","entry":"load_pretrained_model","repo":"deepseek-ai/deepseek-vl2","repo_kind":"official","path":"deepseek_vl2/utils/io.py","file_url":"https://github.com/deepseek-ai/deepseek-vl2/blob/HEAD/deepseek_vl2/utils/io.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8a1727ee6e04ec45"}},{"code_sha256_prefix":"d1fcc287229a99c6","entry":"markdown_to_html_with_syntax_highlight","repo":"deepseek-ai/deepseek-vl2","repo_kind":"official","path":"deepseek_vl2/serve/app_modules/utils.py","file_url":"https://github.com/deepseek-ai/deepseek-vl2/blob/HEAD/deepseek_vl2/serve/app_modules/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d1fcc287229a99c6"}},{"code_sha256_prefix":"6408ebfc6065bf19","entry":"select_best_resolution","repo":"deepseek-ai/deepseek-vl2","repo_kind":"official","path":"deepseek_vl2/models/processing_deepseek_vl_v2.py","file_url":"https://github.com/deepseek-ai/deepseek-vl2/blob/HEAD/deepseek_vl2/models/processing_deepseek_vl_v2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6408ebfc6065bf19"}},{"code_sha256_prefix":"f7fc8e891d83a3e1","entry":"strip_stop_words","repo":"deepseek-ai/deepseek-vl2","repo_kind":"official","path":"deepseek_vl2/serve/app_modules/utils.py","file_url":"https://github.com/deepseek-ai/deepseek-vl2/blob/HEAD/deepseek_vl2/serve/app_modules/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f7fc8e891d83a3e1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}