{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/openvlthinker-an-early-exploration-to-complex","title":"OpenVLThinker: An Early Exploration to Complex Vision-Language Reasoning via Iterative Self-Improvement","arxiv_id":"2503.17352","date":"2025-03-21","proceeding":null,"authors":["Yihe Deng","Hritik Bansal","Fan Yin","Nanyun Peng","Wei Wang","Kai-Wei Chang"],"abstract":"Recent advancements demonstrated by DeepSeek-R1 have shown that complex reasoning abilities in large language models (LLMs), including sophisticated behaviors such as self-verification and self-correction, can be achieved by RL with verifiable rewards and significantly improves model performance on challenging tasks such as AIME. Motivated by these findings, our study investigates whether similar reasoning capabilities can be successfully integrated into large vision-language models (LVLMs) and assesses their impact on challenging multimodal reasoning tasks. We consider an approach that iteratively leverages supervised fine-tuning (SFT) on lightweight training data and Reinforcement Learning (RL) to further improve model generalization. Initially, reasoning capabilities were distilled from pure-text R1 models by generating reasoning steps using high-quality captions of the images sourced from diverse visual datasets. Subsequently, iterative RL training further enhance reasoning skills, with each iteration's RL-improved model generating refined SFT datasets for the next round. This iterative process yielded OpenVLThinker, a LVLM exhibiting consistently improved reasoning performance on challenging benchmarks such as MathVista, MathVerse, and MathVision, demonstrating the potential of our strategy for robust vision-language reasoning. The code, model and data are held at https://github.com/yihedeng9/OpenVLThinker.","url_abs":"https://arxiv.org/abs/2503.17352v1","url_pdf":"https://arxiv.org/pdf/2503.17352v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"openvlthinker-an-early-exploration-to-complex","repo_url":"https://github.com/yihedeng9/openvlthinker","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"multimodal-reasoning","task_name":"Multimodal Reasoning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[{"method_slug":"sft","method_name":"SFT"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.17352","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.17352"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yihedeng9/openvlthinker","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":1,"unverified":20},"by_repo_kind":{"official":{"samples":21,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a91c18341292d50e","entry":"unpad_dataproto","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"EasyR1/verl/protocol.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/EasyR1/verl/protocol.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a91c18341292d50e"}},{"code_sha256_prefix":"8e0e5ee3ad0516dd","entry":"clean_answer","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"v1/utils/clean_answer.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/v1/utils/clean_answer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8e0e5ee3ad0516dd"}},{"code_sha256_prefix":"f041612be625445b","entry":"compute_advantage_return","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"EasyR1/verl/trainer/core_algos.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/EasyR1/verl/trainer/core_algos.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f041612be625445b"}},{"code_sha256_prefix":"5cc7a51b0fa5c955","entry":"download_image","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"v1/generation/generate_reasoning_thinker.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/v1/generation/generate_reasoning_thinker.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5cc7a51b0fa5c955"}},{"code_sha256_prefix":"7688ceba543fa185","entry":"flash_attention_forward","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"EasyR1/verl/models/transformers/flash_attention_utils.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/EasyR1/verl/models/transformers/flash_attention_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7688ceba543fa185"}},{"code_sha256_prefix":"92d15019a67299c8","entry":"get_dataset_config","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"v1/evaluation/eval_openvlthinker.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/v1/evaluation/eval_openvlthinker.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"92d15019a67299c8"}},{"code_sha256_prefix":"095fa22e54f86506","entry":"get_dataset_config","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"v1/evaluation/eval_qwen.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/v1/evaluation/eval_qwen.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"095fa22e54f86506"}},{"code_sha256_prefix":"8b5409f98d042108","entry":"get_dataset_config","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"v1/evaluation/run_evaluation.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/v1/evaluation/run_evaluation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8b5409f98d042108"}},{"code_sha256_prefix":"7633fd09bcf8972a","entry":"get_kl_controller","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"EasyR1/verl/trainer/core_algos.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/EasyR1/verl/trainer/core_algos.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7633fd09bcf8972a"}},{"code_sha256_prefix":"84ec1d8ba06fba36","entry":"get_model_eval_config","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"v1/evaluation/run_evaluation.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/v1/evaluation/run_evaluation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"84ec1d8ba06fba36"}},{"code_sha256_prefix":"ccfc28a818189693","entry":"get_prompt","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"v1/evaluation/verify_mathverse_gpt4.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/v1/evaluation/verify_mathverse_gpt4.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ccfc28a818189693"}},{"code_sha256_prefix":"c90b5e53940a312f","entry":"load_dataset_items","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"v1/evaluation/run_evaluation.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/v1/evaluation/run_evaluation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c90b5e53940a312f"}},{"code_sha256_prefix":"782cbf7f91336a07","entry":"load_image_dataset","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"v1/evaluation/eval_openvlthinker.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/v1/evaluation/eval_openvlthinker.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"782cbf7f91336a07"}},{"code_sha256_prefix":"052c1bc3473227e4","entry":"load_image_dataset","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"v1/evaluation/eval_qwen.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/v1/evaluation/eval_qwen.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"052c1bc3473227e4"}},{"code_sha256_prefix":"c95a4844f5a77402","entry":"load_image_dataset","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"v1/generation/generate_reasoning_thinker.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/v1/generation/generate_reasoning_thinker.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c95a4844f5a77402"}},{"code_sha256_prefix":"21c38014814eae1e","entry":"pad_dataproto_to_divisor","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"EasyR1/verl/protocol.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/EasyR1/verl/protocol.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"21c38014814eae1e"}},{"code_sha256_prefix":"72f505c72eeef3e8","entry":"prepare_fa2_from_position_ids","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"EasyR1/verl/models/transformers/flash_attention_utils.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/EasyR1/verl/models/transformers/flash_attention_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"72f505c72eeef3e8"}},{"code_sha256_prefix":"394985bb469413a6","entry":"process_response","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"v1/evaluation/eval_openvlthinker.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/v1/evaluation/eval_openvlthinker.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"394985bb469413a6"}},{"code_sha256_prefix":"10457e62fda12718","entry":"process_response","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"v1/evaluation/eval_qwen.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/v1/evaluation/eval_qwen.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"10457e62fda12718"}},{"code_sha256_prefix":"339cccf9ceeae43e","entry":"register_adv_estimator","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"EasyR1/verl/trainer/core_algos.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/EasyR1/verl/trainer/core_algos.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"339cccf9ceeae43e"}},{"code_sha256_prefix":"ad658a1d6fe829ba","entry":"union_tensor_dict","repo":"yihedeng9/openvlthinker","repo_kind":"official","path":"EasyR1/verl/protocol.py","file_url":"https://github.com/yihedeng9/openvlthinker/blob/HEAD/EasyR1/verl/protocol.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ad658a1d6fe829ba"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}