Multi-hop Question Answering

Benchmarks archive 2025-07-28

2 leaderboard tables shown for this task, 2 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

4 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 92 papers with code (202 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering
  13 Apr 2021 6 repositories listed Syntology ran 1 of 25 samples · 24 unverified · 4 pointer-only (licence)
  The problem of answering questions using knowledge from pre-trained language models (LMs) and knowledge graphs (KGs) presents two challenges: given a QA context (question and answer choice), methods need to (i) identify…
• Repurposing Entailment for Multi-Hop Question Answering Tasks
  20 Apr 2019 4 repositories listed
  We introduce Multee, a general architecture that can effectively use entailment models for multi-hop QA tasks.
• End-to-End Beam Retrieval for Multi-Hop Question Answering
  17 Aug 2023 3 repositories listed Syntology ran 10 of 13 samples · 3 unverified
  This approach models the multi-hop retrieval process in an end-to-end manner by jointly optimizing an encoder and two classification heads across all hops.
• MuSiQue: Multihop Questions via Single-hop Question Composition
  2 Aug 2021 3 repositories listed Syntology ran 1 of 16 samples · 15 unverified · 1 pointer-only (licence)
  Multihop reasoning remains an elusive goal as existing multihop benchmarks are known to be largely solvable via shortcuts.
• Multi-hop Question Answering via Reasoning Chains
  7 Oct 2019 3 repositories listed
  Our analysis shows the properties of chains that are crucial for high performance: in particular, modeling extraction sequentially is important, as is dealing with each candidate sentence in a context-aware way.
• HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models
  23 May 2024 2 repositories listed Syntology ran 0 of 5 samples · 5 unverified
  In order to thrive in hostile and ever-changing natural environments, mammalian brains evolved to store large amounts of knowledge about the world and continually integrate new information while avoiding catastrophic…
• DeepEdit: Knowledge Editing as Decoding with Constraints
  19 Jan 2024 2 repositories listed
  How to edit the knowledge in multi-step reasoning has become the major challenge in the knowledge editing (KE) of large language models (LLMs).
• MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions
  24 May 2023 2 repositories listed
  The information stored in large language models (LLMs) falls out of date quickly, and retraining from scratch is often not an option.
• Can Language Models Solve Graph Problems in Natural Language?
  17 May 2023 2 repositories listed Syntology ran 2 of 22 samples · 20 unverified
  We then propose Build-a-Graph Prompting and Algorithmic Prompting, two instruction-based approaches to enhance LLMs in solving natural language graph problems.
• Analyzing the Effectiveness of the Underlying Reasoning Tasks in Multi-hop Question Answering
  12 Feb 2023 2 repositories listed
  To explain the predicted answers and evaluate the reasoning abilities of models, several studies have utilized underlying reasoning (UR) tasks in multi-hop question answering (QA) datasets.
• Rethinking Label Smoothing on Multi-hop Question Answering
  19 Dec 2022 2 repositories listed
  Multi-Hop Question Answering (MHQA) is a significant area in question answering, requiring multiple reasoning components, including document retrieval, supporting sentence prediction, and answer span extraction.
• Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings
  1 Jul 2020 2 repositories listed
  In a separate line of research, KG embedding methods have been proposed to reduce KG sparsity by performing missing link prediction.
• HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data
  15 Apr 2020 2 repositories listed Syntology ran 5 of 15 samples · 10 unverified · 2 pointer-only (licence)
  3) a hybrid model that combines heterogeneous information to find the answer.
• Cognitive Graph for Multi-Hop Reading Comprehension at Scale
  14 May 2019 2 repositories listed Syntology ran 5 of 9 samples · 4 unverified
  We propose a new CogQA framework for multi-hop question answering in web-scale documents.
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
  25 Sep 2018 2 repositories listed Syntology ran 3 of 8 samples · 5 unverified
  Existing question answering (QA) datasets fail to train QA systems to perform complex reasoning and provide explanations for answers.
• Commonsense for Generative Multi-Hop Question Answering Tasks
  17 Sep 2018 2 repositories listed Syntology ran 0 of 7 samples · 7 unverified
  We instead focus on a more challenging multi-hop generative task (NarrativeQA), which requires the model to reason, gather, and synthesize disjoint pieces of information within the context to generate an answer.
• Dynamic Chunking and Selection for Reading Comprehension of Ultra-Long Context in Large Language Models
  1 Jun 2025 1 repository listed Syntology ran 9 of 9 samples · 0 unverified · 9 pointer-only (licence)
  To overcome this limitation, we propose a straightforward approach for dynamically separating and selecting chunks of long context, facilitating a more streamlined input for LLMs.
• Scaling External Knowledge Input Beyond Context Windows of LLMs via Multi-Agent Collaboration
  27 May 2025 1 repository listed
  However, the limited context window of LLMs obstructs scaling the amount of external knowledge input, prohibiting further improvement, especially for tasks requiring significant amount of external knowledge.
• MASKSEARCH: A Universal Pre-Training Framework to Enhance Agentic Search Capability
  26 May 2025 1 repository listed
  In the pre-training stage, we introduce the Retrieval Augmented Mask Prediction (RAMP) task, where the model learns to leverage search tools to fill masked spans on a large number of pre-training data, thus acquiring…
• KnowTrace: Bootstrapping Iterative Retrieval-Augmented Generation with Structured Knowledge Tracing
  26 May 2025 1 repository listed
  Recent advances in retrieval-augmented generation (RAG) furnish large language models (LLMs) with iterative retrievals of relevant information to handle complex multi-hop questions.
• Self-Critique Guided Iterative Reasoning for Multi-hop Question Answering
  25 May 2025 1 repository listed
  Recent work explores iterative retrieval to address complex problems.
• SituatedThinker: Grounding LLM Reasoning with Real-World through Situated Thinking
  25 May 2025 1 repository listed
  By utilizing reinforcement learning, SituatedThinker incentivizes deliberate reasoning with the real world to acquire information and feedback, allowing LLMs to surpass their knowledge boundaries and enhance reasoning.
• HopWeaver: Synthesizing Authentic Multi-Hop Questions Across Text Corpora
  21 May 2025 1 repository listed
  Multi-Hop Question Answering (MHQA) is crucial for evaluating the model's capability to integrate information from diverse sources.
• Masking in Multi-hop QA: An Analysis of How Language Models Perform with Context Permutation
  16 May 2025 1 repository listed
  Our study reveals interesting findings as follows: 1) Encoder-decoder models, such as the ones in the Flan-T5 family, generally outperform causal decoder-only LMs in MHQA tasks, despite being significantly smaller in…
• Focus, Merge, Rank: Improved Question Answering Based on Semi-structured Knowledge Bases
  14 May 2025 1 repository listed
  In many real-world settings, machine learning models and interactive systems have access to both structured knowledge, e.
• TreeHop: Generate and Filter Next Query Embeddings Efficiently for Multi-hop Question Answering
  28 Apr 2025 1 repository listed
  Retrieval-augmented generation (RAG) systems face significant challenges in multi-hop question answering (MHQA), where complex queries require synthesizing information across multiple document chunks.
• Collab-RAG: Boosting Retrieval-Augmented Generation for Complex Question Answering via White-Box and Black-Box LLM Collaboration
  7 Apr 2025 1 repository listed Syntology ran 0 of 1 samples · 1 unverified
  Retrieval-Augmented Generation (RAG) systems often struggle to handle multi-hop question-answering tasks accurately due to irrelevant context retrieval and limited complex reasoning capabilities.
• Retrieval-Augmented Generation with Hierarchical Knowledge
  13 Mar 2025 1 repository listed Syntology ran 0 of 11 samples · 11 unverified
  Graph-based Retrieval-Augmented Generation (RAG) methods have significantly enhanced the performance of large language models (LLMs) in domain-specific tasks.
• LevelRAG: Enhancing Retrieval-Augmented Generation with Multi-hop Logic Planning over Rewriting Augmented Searchers
  25 Feb 2025 1 repository listed
  In LevelRAG, the high-level searcher orchestrates the retrieval logic, while the low-level searchers (sparse, web, and dense) refine the queries for optimal retrieval.
• MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledge
  22 Dec 2024 1 repository listed
  To bridge this gap, we introduce MINTQA (Multi-hop Question Answering on New and Tail Knowledge), a comprehensive benchmark to evaluate LLMs' capabilities in multi-hop reasoning across four critical dimensions: question…

Syntology lines on 12 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.