{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2604-20190","title":"WildFireVQA: A Large-Scale Radiometric Thermal VQA Benchmark for Aerial Wildfire Monitoring","arxiv_id":"2604.20190","date":"2026-04-22","proceeding":null,"authors":["Mobin Habibpour","Niloufar Alipour Talemi","John Spodnik","Camren J. Khoury","Fatemeh Afghah"],"abstract":"Wildfire monitoring requires timely, actionable situational awareness from airborne platforms, yet existing aerial visual question answering (VQA) benchmarks do not evaluate wildfire-specific multimodal reasoning grounded in thermal measurements. We introduce WildFireVQA, a large-scale VQA benchmark for aerial wildfire monitoring that integrates RGB imagery with radiometric thermal data. WildFireVQA contains 6,097 RGB-thermal samples, where each sample includes an RGB image, a color-mapped thermal visualization, and a radiometric thermal TIFF, and is paired with 34 questions, yielding a total of 207,298 multiple-choice questions spanning presence and detection, classification, distribution and segmentation, localization and direction, cross-modal reasoning, and flight planning for operational wildfire intelligence. To improve annotation reliability, we combine multimodal large language model (MLLM)-based answer generation with sensor-driven deterministic labeling, manual verification, and intra-frame and inter-frame consistency checks. We further establish a comprehensive evaluation protocol for representative MLLMs under RGB, Thermal, and retrieval-augmented settings using radiometric thermal statistics. Experiments show that across task categories, RGB remains the strongest modality for current models, while retrieved thermal context yields gains for stronger MLLMs, highlighting both the value of temperature-grounded reasoning and the limitations of existing MLLMs in safety-critical wildfire scenarios. The dataset and benchmark code are open-source at https://github.com/mobiiin/WildFire_VQA.","url_abs":"https://arxiv.org/abs/2604.20190","url_pdf":"https://arxiv.org/pdf/2604.20190","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2604.20190","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2604.20190"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/mobiiin/WildFire_VQA","reach":null}],"summary":{"ran":3,"ran_draft_wrong":2,"ran_fixture":1},"by_repo_kind":{"found_in_text":{"samples":6,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"2eb0d8e9f3d6e8df","entry":"build_content_block","repo":"mobiiin/WildFire_VQA","repo_kind":"found_in_text","path":"eval_vlm_checkpoint_icl.py","file_url":"https://github.com/mobiiin/WildFire_VQA/blob/HEAD/eval_vlm_checkpoint_icl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2eb0d8e9f3d6e8df"}},{"code_sha256_prefix":"8dfbf1754dcda381","entry":"build_icl_intro_text","repo":"mobiiin/WildFire_VQA","repo_kind":"found_in_text","path":"eval_vlm_checkpoint_icl.py","file_url":"https://github.com/mobiiin/WildFire_VQA/blob/HEAD/eval_vlm_checkpoint_icl.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8dfbf1754dcda381"}},{"code_sha256_prefix":"b4121550c950dd21","entry":"build_multiturn_message_spec","repo":"mobiiin/WildFire_VQA","repo_kind":"found_in_text","path":"eval_vlm_checkpoint_icl.py","file_url":"https://github.com/mobiiin/WildFire_VQA/blob/HEAD/eval_vlm_checkpoint_icl.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b4121550c950dd21"}},{"code_sha256_prefix":"55ee8356d62ca25f","entry":"build_system_context","repo":"mobiiin/WildFire_VQA","repo_kind":"found_in_text","path":"eval_vlm_checkpoint_icl.py","file_url":"https://github.com/mobiiin/WildFire_VQA/blob/HEAD/eval_vlm_checkpoint_icl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"55ee8356d62ca25f"}},{"code_sha256_prefix":"ab602bd95009928b","entry":"build_task_instruction","repo":"mobiiin/WildFire_VQA","repo_kind":"found_in_text","path":"eval_vlm_checkpoint_icl.py","file_url":"https://github.com/mobiiin/WildFire_VQA/blob/HEAD/eval_vlm_checkpoint_icl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ab602bd95009928b"}},{"code_sha256_prefix":"692d02df18d492e1","entry":"prepend_text_block","repo":"mobiiin/WildFire_VQA","repo_kind":"found_in_text","path":"eval_vlm_checkpoint_icl.py","file_url":"https://github.com/mobiiin/WildFire_VQA/blob/HEAD/eval_vlm_checkpoint_icl.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"692d02df18d492e1"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}