{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2512-10359","title":"Tool-Augmented Spatiotemporal Reasoning for Streamlining Video Question Answering Task","arxiv_id":"2512.10359","date":"2025-12-11","proceeding":"NeurIPS","authors":["Sunqi Fan","Jiashuo Cui","Meng-Hao Guo","Shuojin Yang"],"abstract":"Video Question Answering (VideoQA) task serves as a critical playground for evaluating whether foundation models can effectively perceive, understand, and reason about dynamic real-world scenarios. However, existing Multimodal Large Language Models (MLLMs) struggle with simultaneously modeling spatial relationships within video frames and understanding the causal dynamics of temporal evolution on complex and reasoning-intensive VideoQA task. In this work, we equip MLLM with a comprehensive and extensible Video Toolkit, to enhance MLLM's spatiotemporal reasoning capabilities and ensure the harmony between the quantity and diversity of tools. To better control the tool invocation sequence and avoid toolchain shortcut issues, we propose a Spatiotemporal Reasoning Framework (STAR) that strategically schedules temporal and spatial tools, thereby progressively localizing the key area in the video. Our STAR framework enhances GPT-4o using lightweight tools, achieving an 8.2% gain on VideoMME and 4.6% on LongVideoBench. We believe that our proposed Video Toolkit and STAR framework make an important step towards building autonomous and intelligent video analysis assistants. The code is publicly available at https://github.com/fansunqi/VideoTool.","url_abs":"https://arxiv.org/abs/2512.10359","url_pdf":"https://arxiv.org/pdf/2512.10359","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2512.10359","atlas_url":"https://app.syntology.ai/?focus=2512.10359","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2512.10359"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/fansunqi/VideoTool","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":10},"by_repo_kind":{"found_in_text":{"samples":10,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3d3b5a6ede94a143","entry":"check_videomme_option","repo":"fansunqi/VideoTool","repo_kind":"found_in_text","path":"dataset.py","file_url":"https://github.com/fansunqi/VideoTool/blob/HEAD/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d3b5a6ede94a143"}},{"code_sha256_prefix":"f1ecd2f415293840","entry":"check_videomme_option","repo":"fansunqi/VideoTool","repo_kind":"found_in_text","path":"testcases/extract_testcase_from_videomme.py","file_url":"https://github.com/fansunqi/VideoTool/blob/HEAD/testcases/extract_testcase_from_videomme.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f1ecd2f415293840"}},{"code_sha256_prefix":"42677134b9a94956","entry":"get_dataset","repo":"fansunqi/VideoTool","repo_kind":"found_in_text","path":"dataset.py","file_url":"https://github.com/fansunqi/VideoTool/blob/HEAD/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"42677134b9a94956"}},{"code_sha256_prefix":"11b4ab61b5277e1f","entry":"get_tool_category","repo":"fansunqi/VideoTool","repo_kind":"found_in_text","path":"star_prompts.py","file_url":"https://github.com/fansunqi/VideoTool/blob/HEAD/star_prompts.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"11b4ab61b5277e1f"}},{"code_sha256_prefix":"2a7e37b022633a01","entry":"get_tool_instances","repo":"fansunqi/VideoTool","repo_kind":"found_in_text","path":"run_single_video.py","file_url":"https://github.com/fansunqi/VideoTool/blob/HEAD/run_single_video.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a7e37b022633a01"}},{"code_sha256_prefix":"6f99f335c87103a2","entry":"get_video_info","repo":"fansunqi/VideoTool","repo_kind":"found_in_text","path":"visible_frames.py","file_url":"https://github.com/fansunqi/VideoTool/blob/HEAD/visible_frames.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6f99f335c87103a2"}},{"code_sha256_prefix":"bffb29c594b85f95","entry":"identify_tools","repo":"fansunqi/VideoTool","repo_kind":"found_in_text","path":"reasoning.py","file_url":"https://github.com/fansunqi/VideoTool/blob/HEAD/reasoning.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bffb29c594b85f95"}},{"code_sha256_prefix":"bc33bf64eaeb089c","entry":"load_cache","repo":"fansunqi/VideoTool","repo_kind":"found_in_text","path":"util.py","file_url":"https://github.com/fansunqi/VideoTool/blob/HEAD/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc33bf64eaeb089c"}},{"code_sha256_prefix":"008a66014da94a6f","entry":"load_llava_model","repo":"fansunqi/VideoTool","repo_kind":"found_in_text","path":"util.py","file_url":"https://github.com/fansunqi/VideoTool/blob/HEAD/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"008a66014da94a6f"}},{"code_sha256_prefix":"e37d7eb84d2fafdf","entry":"strip_option_prefix","repo":"fansunqi/VideoTool","repo_kind":"found_in_text","path":"testcases/extract_testcase_from_videomme.py","file_url":"https://github.com/fansunqi/VideoTool/blob/HEAD/testcases/extract_testcase_from_videomme.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e37d7eb84d2fafdf"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_api"},"syntology_extracted_results":null}