{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/complextempqa-a-large-scale-dataset-for","title":"ComplexTempQA: A Large-Scale Dataset for Complex Temporal Question Answering","arxiv_id":"2406.04866","date":"2024-06-07","proceeding":null,"authors":["Raphael Gruber","Abdelrahman Abdallah","Michael Färber","Adam Jatowt"],"abstract":"We introduce ComplexTempQA, a large-scale dataset consisting of over 100 million question-answer pairs designed to tackle the challenges in temporal question answering. ComplexTempQA significantly surpasses existing benchmarks like HOTPOTQA, TORQUE, and TEQUILA in scale and scope. Utilizing data from Wikipedia and Wikidata, the dataset covers questions spanning over two decades and offers an unmatched breadth of topics. We introduce a unique taxonomy that categorizes questions as attributes, comparisons, and counting questions, each revolving around events, entities, and time periods. One standout feature of ComplexTempQA is the high complexity of its questions, which demand effective capabilities for answering such as across-time comparison, temporal aggregation, and multi-hop reasoning involving temporal event ordering and entity recognition. Additionally, each question is accompanied by detailed metadata, including specific time scopes, allowing for comprehensive evaluation and enhancement of the temporal reasoning abilities of large language models. ComplexTempQA serves both as a testing ground for developing sophisticated AI models and as a foundation for advancing research in question answering, information retrieval, and language understanding.","url_abs":"https://arxiv.org/abs/2406.04866v2","url_pdf":"https://arxiv.org/pdf/2406.04866v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"complextempqa-a-large-scale-dataset-for","repo_url":"https://github.com/datascienceuibk/complextempqa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"information-retrieval","task_name":"Information Retrieval"},{"task_slug":"question-answering","task_name":"Question Answering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.04866","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.04866"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/datascienceuibk/complextempqa","reach":null}],"summary":{"ran_draft_wrong":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f6361ab8b0723013","entry":"SPARQLGenerator","repo":"datascienceuibk/complextempqa","repo_kind":"official","path":"src/QuestionGeneration/SPARQLGenerator2.py","file_url":"https://github.com/datascienceuibk/complextempqa/blob/HEAD/src/QuestionGeneration/SPARQLGenerator2.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f6361ab8b0723013"}},{"code_sha256_prefix":"f29c51a54ccffc35","entry":"SPARQLGeneratorCount","repo":"datascienceuibk/complextempqa","repo_kind":"official","path":"src/QuestionGeneration/SPARQLGenerator2.py","file_url":"https://github.com/datascienceuibk/complextempqa/blob/HEAD/src/QuestionGeneration/SPARQLGenerator2.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f29c51a54ccffc35"}},{"code_sha256_prefix":"8f89f44dc4aff5a6","entry":"createSingleStatement","repo":"datascienceuibk/complextempqa","repo_kind":"official","path":"src/QuestionGeneration/SPARQLGenerator2.py","file_url":"https://github.com/datascienceuibk/complextempqa/blob/HEAD/src/QuestionGeneration/SPARQLGenerator2.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"none","behaviour_fingerprint":true,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8f89f44dc4aff5a6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}