{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cityeqa-a-hierarchical-llm-agent-on-embodied","title":"CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Space","arxiv_id":"2502.12532","date":"2025-02-18","proceeding":null,"authors":["Yong Zhao","Kai Xu","Zhengqiu Zhu","Yue Hu","Zhiheng Zheng","Yingfeng Chen","Yatai Ji","Chen Gao","Yong Li","Jincai Huang"],"abstract":"Embodied Question Answering (EQA) has primarily focused on indoor environments, leaving the complexities of urban settings - spanning environment, action, and perception - largely unexplored. To bridge this gap, we introduce CityEQA, a new task where an embodied agent answers open-vocabulary questions through active exploration in dynamic city spaces. To support this task, we present CityEQA-EC, the first benchmark dataset featuring 1,412 human-annotated tasks across six categories, grounded in a realistic 3D urban simulator. Moreover, we propose Planner-Manager-Actor (PMA), a novel agent tailored for CityEQA. PMA enables long-horizon planning and hierarchical task execution: the Planner breaks down the question answering into sub-tasks, the Manager maintains an object-centric cognitive map for spatial reasoning during the process control, and the specialized Actors handle navigation, exploration, and collection sub-tasks. Experiments demonstrate that PMA achieves 60.7% of human-level answering accuracy, significantly outperforming frontier-based baselines. While promising, the performance gap compared to humans highlights the need for enhanced visual reasoning in CityEQA. This work paves the way for future advancements in urban spatial intelligence. Dataset and code are available at https://github.com/BiluYong/CityEQA.git.","url_abs":"https://arxiv.org/abs/2502.12532v2","url_pdf":"https://arxiv.org/pdf/2502.12532v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cityeqa-a-hierarchical-llm-agent-on-embodied","repo_url":"https://github.com/biluyong/cityeqa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null},{"paper_slug":"cityeqa-a-hierarchical-llm-agent-on-embodied","repo_url":"https://github.com/tsinghua-fib-lab/CityEQA","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"embodied-question-answering","task_name":"Embodied Question Answering"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"spatial-reasoning","task_name":"Spatial Reasoning"},{"task_slug":"visual-reasoning","task_name":"Visual Reasoning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2502.12532","atlas_url":"https://app.syntology.ai/?focus=2502.12532","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.12532"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tsinghua-fib-lab/CityEQA","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/biluyong/cityeqa","reach":null}],"summary":{"ran":2,"unverified":2},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"3658f9d5e4456cc1","entry":"Collector","repo":"biluyong/cityeqa","repo_kind":"official","path":"Agent/manager.py","file_url":"https://github.com/biluyong/cityeqa/blob/HEAD/Agent/manager.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3658f9d5e4456cc1"}},{"code_sha256_prefix":"e607b6521052181a","entry":"Explorer","repo":"biluyong/cityeqa","repo_kind":"official","path":"Agent/manager.py","file_url":"https://github.com/biluyong/cityeqa/blob/HEAD/Agent/manager.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e607b6521052181a"}},{"code_sha256_prefix":"36217a6593c554c1","entry":"Manager","repo":"biluyong/cityeqa","repo_kind":"official","path":"Agent/manager.py","file_url":"https://github.com/biluyong/cityeqa/blob/HEAD/Agent/manager.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"36217a6593c554c1"}},{"code_sha256_prefix":"7893eba90c7709f6","entry":"Navigator","repo":"biluyong/cityeqa","repo_kind":"official","path":"Agent/manager.py","file_url":"https://github.com/biluyong/cityeqa/blob/HEAD/Agent/manager.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7893eba90c7709f6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}