{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/play-to-generalize-learning-to-reason-through","title":"Play to Generalize: Learning to Reason Through Game Play","arxiv_id":"2506.08011","date":"2025-06-09","proceeding":null,"authors":["Yunfei Xie","Yinsong Ma","Shiyi Lan","Alan Yuille","Junfei Xiao","Chen Wei"],"abstract":"Developing generalizable reasoning capabilities in multimodal large language models (MLLMs) remains challenging. Motivated by cognitive science literature suggesting that gameplay promotes transferable cognitive skills, we propose a novel post-training paradigm, Visual Game Learning, or ViGaL, where MLLMs develop out-of-domain generalization of multimodal reasoning through playing arcade-like games. Specifically, we show that post-training a 7B-parameter MLLM via reinforcement learning (RL) on simple arcade-like games, e.g. Snake, significantly enhances its downstream performance on multimodal math benchmarks like MathVista, and on multi-discipline questions like MMMU, without seeing any worked solutions, equations, or diagrams during RL, suggesting the capture of transferable reasoning skills. Remarkably, our model outperforms specialist models tuned on multimodal reasoning data in multimodal reasoning benchmarks, while preserving the base model's performance on general visual benchmarks, a challenge where specialist models often fall short. Our findings suggest a new post-training paradigm: synthetic, rule-based games can serve as controllable and scalable pre-text tasks that unlock generalizable multimodal reasoning abilities in MLLMs.","url_abs":"https://arxiv.org/abs/2506.08011v1","url_pdf":"https://arxiv.org/pdf/2506.08011v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"play-to-generalize-learning-to-reason-through","repo_url":"https://github.com/yunfeixie233/vigal","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"domain-generalization","task_name":"Domain Generalization"},{"task_slug":"math","task_name":"Math"},{"task_slug":"multimodal-reasoning","task_name":"Multimodal Reasoning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[{"method_slug":"base","method_name":"BASE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2506.08011","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.08011"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yunfeixie233/vigal","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":12},"by_repo_kind":{"official":{"samples":12,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f8a7afabd4d22b07","entry":"build_mathverse_gpt4_extract_prompt","repo":"yunfeixie233/vigal","repo_kind":"official","path":"eval/mathverse/extract_calculate.py","file_url":"https://github.com/yunfeixie233/vigal/blob/HEAD/eval/mathverse/extract_calculate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f8a7afabd4d22b07"}},{"code_sha256_prefix":"566d6789169e7b2c","entry":"build_mathverse_gpt4_score_prompt","repo":"yunfeixie233/vigal","repo_kind":"official","path":"eval/mathverse/extract_calculate.py","file_url":"https://github.com/yunfeixie233/vigal/blob/HEAD/eval/mathverse/extract_calculate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"566d6789169e7b2c"}},{"code_sha256_prefix":"e2935c878a3bfb63","entry":"can_infer_option","repo":"yunfeixie233/vigal","repo_kind":"official","path":"eval/mathvision/extract_calculate.py","file_url":"https://github.com/yunfeixie233/vigal/blob/HEAD/eval/mathvision/extract_calculate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e2935c878a3bfb63"}},{"code_sha256_prefix":"59ef20be70813be5","entry":"can_infer_text","repo":"yunfeixie233/vigal","repo_kind":"official","path":"eval/mathvista/extract_calculate.py","file_url":"https://github.com/yunfeixie233/vigal/blob/HEAD/eval/mathvista/extract_calculate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"59ef20be70813be5"}},{"code_sha256_prefix":"95084b4b49902e3d","entry":"compute_approx_kl","repo":"yunfeixie233/vigal","repo_kind":"official","path":"openrlhf/models/utils.py","file_url":"https://github.com/yunfeixie233/vigal/blob/HEAD/openrlhf/models/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"95084b4b49902e3d"}},{"code_sha256_prefix":"0d27f3b8c425d79f","entry":"compute_reward","repo":"yunfeixie233/vigal","repo_kind":"official","path":"openrlhf/models/utils.py","file_url":"https://github.com/yunfeixie233/vigal/blob/HEAD/openrlhf/models/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0d27f3b8c425d79f"}},{"code_sha256_prefix":"23c006771508ace1","entry":"convert_ring_attn_params","repo":"yunfeixie233/vigal","repo_kind":"official","path":"openrlhf/models/ring_attn_utils.py","file_url":"https://github.com/yunfeixie233/vigal/blob/HEAD/openrlhf/models/ring_attn_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"23c006771508ace1"}},{"code_sha256_prefix":"2d84175fa37aa178","entry":"get_chat_response","repo":"yunfeixie233/vigal","repo_kind":"official","path":"eval/mathverse/extract_calculate.py","file_url":"https://github.com/yunfeixie233/vigal/blob/HEAD/eval/mathverse/extract_calculate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2d84175fa37aa178"}},{"code_sha256_prefix":"7b812ccf830749f3","entry":"get_chat_response","repo":"yunfeixie233/vigal","repo_kind":"official","path":"eval/mathvision/extract_calculate.py","file_url":"https://github.com/yunfeixie233/vigal/blob/HEAD/eval/mathvision/extract_calculate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7b812ccf830749f3"}},{"code_sha256_prefix":"0612b61574b81472","entry":"log_probs_from_logits","repo":"yunfeixie233/vigal","repo_kind":"official","path":"openrlhf/models/utils.py","file_url":"https://github.com/yunfeixie233/vigal/blob/HEAD/openrlhf/models/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0612b61574b81472"}},{"code_sha256_prefix":"b26627fa753393a8","entry":"pad_sequences","repo":"yunfeixie233/vigal","repo_kind":"official","path":"openrlhf/models/ring_attn_utils.py","file_url":"https://github.com/yunfeixie233/vigal/blob/HEAD/openrlhf/models/ring_attn_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b26627fa753393a8"}},{"code_sha256_prefix":"e93117d99aa6b210","entry":"reset_ring_attn_position_ids","repo":"yunfeixie233/vigal","repo_kind":"official","path":"openrlhf/models/ring_attn_utils.py","file_url":"https://github.com/yunfeixie233/vigal/blob/HEAD/openrlhf/models/ring_attn_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e93117d99aa6b210"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}