{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2602-22859","title":"From Blind Spots to Gains: Diagnostic-Driven Iterative Training for Large Multimodal Models","arxiv_id":"2602.22859","date":"2026-02-26","proceeding":"ICML","authors":["Hongrui Jia","Chaoya Jiang","Yongrui Heng","Shikun Zhang","Wei Ye"],"abstract":"As Large Multimodal Models (LMMs) scale up and reinforcement learning (RL) methods mature, LMMs have made notable progress in complex reasoning and decision making. Yet training still relies on static data and fixed recipes, making it difficult to diagnose capability blind spots or provide dynamic, targeted reinforcement. Motivated by findings that test driven error exposure and feedback based correction outperform repetitive practice, we propose Diagnostic-driven Progressive Evolution (DPE), a spiral loop where diagnosis steers data generation and reinforcement, and each iteration re-diagnoses the updated model to drive the next round of targeted improvement. DPE has two key components. First, multiple agents annotate and quality control massive unlabeled multimodal data, using tools such as web search and image editing to produce diverse, realistic samples. Second, DPE attributes failures to specific weaknesses, dynamically adjusts the data mixture, and guides agents to generate weakness focused data for targeted reinforcement. Experiments on Qwen3-VL-8B-Instruct and Qwen2.5-VL-7B-Instruct show stable, continual gains across eleven benchmarks, indicating DPE as a scalable paradigm for continual LMM training under open task distributions. Our code, models, and data are publicly available at https://github.com/hongruijia/DPE.","url_abs":"https://arxiv.org/abs/2602.22859","url_pdf":"https://arxiv.org/pdf/2602.22859","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2602.22859","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2602.22859"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/hongruijia/DPE","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":9},"by_repo_kind":{"found_in_text":{"samples":10,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"32cbd7aa87e2f64b","entry":"merge_by_placement","repo":"hongruijia/DPE","repo_kind":"found_in_text","path":"scripts_Qwen-VL-7B/model_merger.py","file_url":"https://github.com/hongruijia/DPE/blob/HEAD/scripts_Qwen-VL-7B/model_merger.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"32cbd7aa87e2f64b"}},{"code_sha256_prefix":"cb8e8922a74cff4b","entry":"add_default_scores","repo":"hongruijia/DPE","repo_kind":"found_in_text","path":"question_evaluate/merge_and_convert.py","file_url":"https://github.com/hongruijia/DPE/blob/HEAD/question_evaluate/merge_and_convert.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cb8e8922a74cff4b"}},{"code_sha256_prefix":"c4539c57a9235a68","entry":"aggregate_by_imagetype","repo":"hongruijia/DPE","repo_kind":"found_in_text","path":"question_evaluate/regenerate_prompt_with_llm.py","file_url":"https://github.com/hongruijia/DPE/blob/HEAD/question_evaluate/regenerate_prompt_with_llm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c4539c57a9235a68"}},{"code_sha256_prefix":"0267e29344d42128","entry":"analyze_weakness_with_llm","repo":"hongruijia/DPE","repo_kind":"found_in_text","path":"question_evaluate/regenerate_prompt_with_llm.py","file_url":"https://github.com/hongruijia/DPE/blob/HEAD/question_evaluate/regenerate_prompt_with_llm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0267e29344d42128"}},{"code_sha256_prefix":"8db1299d2238c123","entry":"b64_to_image","repo":"hongruijia/DPE","repo_kind":"found_in_text","path":"question_evaluate/evaluate.py","file_url":"https://github.com/hongruijia/DPE/blob/HEAD/question_evaluate/evaluate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8db1299d2238c123"}},{"code_sha256_prefix":"68177b1800fe39d4","entry":"b64_to_image","repo":"hongruijia/DPE","repo_kind":"found_in_text","path":"question_evaluate/generate_answers_parallel.py","file_url":"https://github.com/hongruijia/DPE/blob/HEAD/question_evaluate/generate_answers_parallel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"68177b1800fe39d4"}},{"code_sha256_prefix":"a358bb64d8f0c7a4","entry":"convert_to_parquet","repo":"hongruijia/DPE","repo_kind":"found_in_text","path":"question_evaluate/merge_and_convert.py","file_url":"https://github.com/hongruijia/DPE/blob/HEAD/question_evaluate/merge_and_convert.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a358bb64d8f0c7a4"}},{"code_sha256_prefix":"6c3a03cdc78f51d2","entry":"decode_base64_image","repo":"hongruijia/DPE","repo_kind":"found_in_text","path":"question_evaluate/score_questions.py","file_url":"https://github.com/hongruijia/DPE/blob/HEAD/question_evaluate/score_questions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6c3a03cdc78f51d2"}},{"code_sha256_prefix":"99097aa76bc784c2","entry":"generate_weakness_prompt","repo":"hongruijia/DPE","repo_kind":"found_in_text","path":"question_evaluate/regenerate_prompt_with_llm.py","file_url":"https://github.com/hongruijia/DPE/blob/HEAD/question_evaluate/regenerate_prompt_with_llm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"99097aa76bc784c2"}},{"code_sha256_prefix":"532a063c4d397d63","entry":"merge_json_files","repo":"hongruijia/DPE","repo_kind":"found_in_text","path":"question_evaluate/merge_and_convert.py","file_url":"https://github.com/hongruijia/DPE/blob/HEAD/question_evaluate/merge_and_convert.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"532a063c4d397d63"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}