{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/living-in-the-moment-can-large-language","title":"Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning?","arxiv_id":"2406.09072","date":"2024-06-13","proceeding":null,"authors":["Zhaochen Su","Juntao Li","Jun Zhang","Tong Zhu","Xiaoye Qu","Pan Zhou","Yan Bowen","Yu Cheng","Min Zhang"],"abstract":"Temporal reasoning is fundamental for large language models (LLMs) to comprehend the world. Current temporal reasoning datasets are limited to questions about single or isolated events, falling short in mirroring the realistic temporal characteristics involving concurrent nature and intricate temporal interconnections. In this paper, we introduce CoTempQA, a comprehensive co-temporal Question Answering (QA) benchmark containing four co-temporal scenarios (Equal, Overlap, During, Mix) with 4,748 samples for evaluating the co-temporal comprehension and reasoning abilities of LLMs. Our extensive experiments reveal a significant gap between the performance of current LLMs and human-level reasoning on CoTempQA tasks. Even when enhanced with Chain of Thought (CoT) methodologies, models consistently struggle with our task. In our preliminary exploration, we discovered that mathematical reasoning plays a significant role in handling co-temporal events and proposed a strategy to boost LLMs' co-temporal reasoning from a mathematical perspective. We hope that our CoTempQA datasets will encourage further advancements in improving the co-temporal reasoning capabilities of LLMs. Our code is available at https://github.com/zhaochen0110/Cotempqa.","url_abs":"https://arxiv.org/abs/2406.09072v1","url_pdf":"https://arxiv.org/pdf/2406.09072v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"living-in-the-moment-can-large-language","repo_url":"https://github.com/zhaochen0110/cotempqa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"mathematical-reasoning","task_name":"Mathematical Reasoning"},{"task_slug":"question-answering","task_name":"Question Answering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.09072","atlas_url":"https://app.syntology.ai/?focus=2406.09072","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.09072"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhaochen0110/cotempqa","reach":{"status":"ok"}}],"summary":{"ran":12,"unverified":1},"by_repo_kind":{"official":{"samples":13,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":13,"samples":[{"code_sha256_prefix":"437f492520e9c75b","entry":"complete_time","repo":"zhaochen0110/cotempqa","repo_kind":"official","path":"data_construct/construct_cotempqa.py","file_url":"https://github.com/zhaochen0110/cotempqa/blob/HEAD/data_construct/construct_cotempqa.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"437f492520e9c75b"}},{"code_sha256_prefix":"56249fb66fc01d89","entry":"convert_time","repo":"zhaochen0110/cotempqa","repo_kind":"official","path":"data_construct/classify_data.py","file_url":"https://github.com/zhaochen0110/cotempqa/blob/HEAD/data_construct/classify_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"56249fb66fc01d89"}},{"code_sha256_prefix":"17bca88667af7592","entry":"convert_time","repo":"zhaochen0110/cotempqa","repo_kind":"official","path":"data_construct/structured_to_query.py","file_url":"https://github.com/zhaochen0110/cotempqa/blob/HEAD/data_construct/structured_to_query.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"17bca88667af7592"}},{"code_sha256_prefix":"2559f828ec515684","entry":"custom_sort","repo":"zhaochen0110/cotempqa","repo_kind":"official","path":"data_construct/construct_cotempqa.py","file_url":"https://github.com/zhaochen0110/cotempqa/blob/HEAD/data_construct/construct_cotempqa.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2559f828ec515684"}},{"code_sha256_prefix":"1d5bd4c4648b1e9d","entry":"exact_match_score","repo":"zhaochen0110/cotempqa","repo_kind":"official","path":"config.py","file_url":"https://github.com/zhaochen0110/cotempqa/blob/HEAD/config.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1d5bd4c4648b1e9d"}},{"code_sha256_prefix":"a04566d8526cd8fc","entry":"get_prompts","repo":"zhaochen0110/cotempqa","repo_kind":"official","path":"config.py","file_url":"https://github.com/zhaochen0110/cotempqa/blob/HEAD/config.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a04566d8526cd8fc"}},{"code_sha256_prefix":"8d7a6f157df86980","entry":"judge_imply","repo":"zhaochen0110/cotempqa","repo_kind":"official","path":"data_construct/construct_cotempqa.py","file_url":"https://github.com/zhaochen0110/cotempqa/blob/HEAD/data_construct/construct_cotempqa.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8d7a6f157df86980"}},{"code_sha256_prefix":"e5b347d9d2887961","entry":"normalize_answer","repo":"zhaochen0110/cotempqa","repo_kind":"official","path":"config.py","file_url":"https://github.com/zhaochen0110/cotempqa/blob/HEAD/config.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e5b347d9d2887961"}},{"code_sha256_prefix":"fd241f0a32ac51bb","entry":"parse_date","repo":"zhaochen0110/cotempqa","repo_kind":"official","path":"data_construct/facts_to_raw_data.py","file_url":"https://github.com/zhaochen0110/cotempqa/blob/HEAD/data_construct/facts_to_raw_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fd241f0a32ac51bb"}},{"code_sha256_prefix":"f4d45cae17843b23","entry":"read_qid_names","repo":"zhaochen0110/cotempqa","repo_kind":"official","path":"data_construct/structured_to_query.py","file_url":"https://github.com/zhaochen0110/cotempqa/blob/HEAD/data_construct/structured_to_query.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f4d45cae17843b23"}},{"code_sha256_prefix":"c777c504f6849d65","entry":"read_templates","repo":"zhaochen0110/cotempqa","repo_kind":"official","path":"data_construct/facts_to_raw_data.py","file_url":"https://github.com/zhaochen0110/cotempqa/blob/HEAD/data_construct/facts_to_raw_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c777c504f6849d65"}},{"code_sha256_prefix":"5f2d8ff589bcc638","entry":"time_relation","repo":"zhaochen0110/cotempqa","repo_kind":"official","path":"data_construct/classify_data.py","file_url":"https://github.com/zhaochen0110/cotempqa/blob/HEAD/data_construct/classify_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5f2d8ff589bcc638"}},{"code_sha256_prefix":"254062a7abf992da","entry":"classify_data","repo":"zhaochen0110/cotempqa","repo_kind":"official","path":"data_construct/classify_data.py","file_url":"https://github.com/zhaochen0110/cotempqa/blob/HEAD/data_construct/classify_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"254062a7abf992da"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}