Papers › Transformer-XH: Multi-Evidence Reasoning with eXtra Hop Attention
Transformer-XH: Multi-Evidence Reasoning with eXtra Hop Attention
Chen Zhao, Chenyan Xiong, Corby Rosset, Xia Song, Paul Bennett, Saurabh Tiwary
Transformers have achieved new heights modeling natural language as a sequence of text tokens. However, in many real world scenarios, textual data inherently exhibits structures beyond a linear sequence such as trees and graphs; many tasks require reasoning with evidence scattered across multiple pieces of texts. This paper presents Transformer-XH, which uses eXtra Hop attention to enable intrinsic modeling of structured texts in a fully data-driven way. Its new attention mechanism naturally “hops” across the connected text sequences in addition to attending over tokens within each sequence. Thus, Transformer-XH better conducts joint multi-evidence reasoning by propagating information between documents and constructing global contextualized representations. On multi-hop question answering, Transformer-XH leads to a simpler multi-hop QA system which outperforms previous state-of-the-art on the HotpotQA FullWiki setting. On FEVER fact verification, applying Transformer-XH provides state-of-the-art accuracy and excels on claims whose verification requires multiple evidence.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Question Answering | HotpotQA | Transformer-XH-final | ANS-EM | 0.516 | #42 of 72 | Archive leaderboard | report |
| Question Answering | HotpotQA | Transformer-XH-final | ANS-F1 | 0.641 | #42 of 72 | Archive leaderboard | report |
| Question Answering | HotpotQA | Transformer-XH-final | JOINT-EM | 0.261 | #42 of 72 | Archive leaderboard | report |
| Question Answering | HotpotQA | Transformer-XH-final | JOINT-F1 | 0.513 | #42 of 72 | Archive leaderboard | report |
| Question Answering | HotpotQA | Transformer-XH-final | SUP-EM | 0.409 | #42 of 72 | Archive leaderboard | report |
| Question Answering | HotpotQA | Transformer-XH-final | SUP-F1 | 0.714 | #42 of 72 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections