{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/spatial-mental-modeling-from-limited-views","title":"Spatial Mental Modeling from Limited Views","arxiv_id":"2506.21458","date":"2025-06-26","proceeding":null,"authors":["Baiqiao Yin","Qineng Wang","Pingyue Zhang","Jianshu Zhang","Kangrui Wang","Zihan Wang","Jieyu Zhang","Keshigeyan Chandrasegaran","Han Liu","Ranjay Krishna","Saining Xie","Manling Li","Jiajun Wu","Li Fei-Fei"],"abstract":"Can Vision Language Models (VLMs) imagine the full scene from just a few views, like humans do? Humans form spatial mental models, internal representations of unseen space, to reason about layout, perspective, and motion. Our new MindCube benchmark with 21,154 questions across 3,268 images exposes this critical gap, where existing VLMs exhibit near-random performance. Using MindCube, we systematically evaluate how well VLMs build robust spatial mental models through representing positions (cognitive mapping), orientations (perspective-taking), and dynamics (mental simulation for \"what-if\" movements). We then explore three approaches to help VLMs approximate spatial mental models, including unseen intermediate views, natural language reasoning chains, and cognitive maps. The significant improvement comes from a synergistic approach, \"map-then-reason\", that jointly trains the model to first generate a cognitive map and then reason upon it. By training models to reason over these internal maps, we boosted accuracy from 37.8% to 60.8% (+23.0%). Adding reinforcement learning pushed performance even further to 70.7% (+32.9%). Our key insight is that such scaffolding of spatial mental models, actively constructing and utilizing internal structured spatial representations with flexible reasoning processes, significantly improves understanding of unobservable space.","url_abs":"https://arxiv.org/abs/2506.21458v1","url_pdf":"https://arxiv.org/pdf/2506.21458v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"spatial-mental-modeling-from-limited-views","repo_url":"https://github.com/mll-lab-nu/mindcube","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"spatial-mental-modeling-from-limited-views","repo_url":"https://github.com/qinengwang-aiden/mindcube","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2506.21458","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.21458"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mll-lab-nu/mindcube","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qinengwang-aiden/mindcube","reach":{"status":"ok"}}],"summary":{"unverified":11},"by_repo_kind":{"official":{"samples":11,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"87a8ebeed7f3f035","entry":"auto_detect_task_type","repo":"mll-lab-nu/mindcube","repo_kind":"official","path":"src/prompt_generation/generators.py","file_url":"https://github.com/mll-lab-nu/mindcube/blob/HEAD/src/prompt_generation/generators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"87a8ebeed7f3f035"}},{"code_sha256_prefix":"702bc0399aaa7f03","entry":"auto_evaluate","repo":"mll-lab-nu/mindcube","repo_kind":"official","path":"src/evaluation/evaluator.py","file_url":"https://github.com/mll-lab-nu/mindcube/blob/HEAD/src/evaluation/evaluator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"702bc0399aaa7f03"}},{"code_sha256_prefix":"70ddecd293062b98","entry":"convert_to_qwen_format","repo":"mll-lab-nu/mindcube","repo_kind":"official","path":"src/scaffold_curation/formatters.py","file_url":"https://github.com/mll-lab-nu/mindcube/blob/HEAD/src/scaffold_curation/formatters.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"70ddecd293062b98"}},{"code_sha256_prefix":"9ede3fd776f54f06","entry":"convert_to_training_format","repo":"mll-lab-nu/mindcube","repo_kind":"official","path":"src/scaffold_curation/formatters.py","file_url":"https://github.com/mll-lab-nu/mindcube/blob/HEAD/src/scaffold_curation/formatters.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9ede3fd776f54f06"}},{"code_sha256_prefix":"2060b70a5dd8ad5c","entry":"evaluate","repo":"mll-lab-nu/mindcube","repo_kind":"official","path":"src/evaluation/evaluator.py","file_url":"https://github.com/mll-lab-nu/mindcube/blob/HEAD/src/evaluation/evaluator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2060b70a5dd8ad5c"}},{"code_sha256_prefix":"771bba39c3faa929","entry":"format_cogmap_json","repo":"mll-lab-nu/mindcube","repo_kind":"official","path":"src/scaffold_curation/formatters.py","file_url":"https://github.com/mll-lab-nu/mindcube/blob/HEAD/src/scaffold_curation/formatters.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"771bba39c3faa929"}},{"code_sha256_prefix":"44a622ddfb307ffd","entry":"get_default_scaffold_output_path","repo":"mll-lab-nu/mindcube","repo_kind":"official","path":"src/scaffold_curation/processors.py","file_url":"https://github.com/mll-lab-nu/mindcube/blob/HEAD/src/scaffold_curation/processors.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"44a622ddfb307ffd"}},{"code_sha256_prefix":"d76ae7075d2b7baa","entry":"get_formatter","repo":"mll-lab-nu/mindcube","repo_kind":"official","path":"src/training/data_formatters.py","file_url":"https://github.com/mll-lab-nu/mindcube/blob/HEAD/src/training/data_formatters.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d76ae7075d2b7baa"}},{"code_sha256_prefix":"19a3c1aeee42e9be","entry":"get_scaffold_directory_structure","repo":"mll-lab-nu/mindcube","repo_kind":"official","path":"src/scaffold_curation/processors.py","file_url":"https://github.com/mll-lab-nu/mindcube/blob/HEAD/src/scaffold_curation/processors.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19a3c1aeee42e9be"}},{"code_sha256_prefix":"6af73b3efc06d0f8","entry":"get_template","repo":"mll-lab-nu/mindcube","repo_kind":"official","path":"src/prompt_generation/templates.py","file_url":"https://github.com/mll-lab-nu/mindcube/blob/HEAD/src/prompt_generation/templates.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6af73b3efc06d0f8"}},{"code_sha256_prefix":"f18780a339ddf735","entry":"validate_scaffold_data","repo":"mll-lab-nu/mindcube","repo_kind":"official","path":"src/prompt_generation/processors.py","file_url":"https://github.com/mll-lab-nu/mindcube/blob/HEAD/src/prompt_generation/processors.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f18780a339ddf735"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}