Papers › FiE: Building a Global Probability Space by Leveraging Early Fusion in Encoder for...
FiE: Building a Global Probability Space by Leveraging Early Fusion in Encoder for Open-Domain Question Answering
Akhil Kedia, Mohd Abbas Zaidi, Haejun Lee
Generative models have recently started to outperform extractive models in Open Domain Question Answering, largely by leveraging their decoder to attend over multiple encoded passages and combining their information. However, generative models tend to be larger than extractive models due to the need for a decoder, run slower during inference due to auto-regressive decoder beam search, and their generated output often suffers from hallucinations. We propose to extend transformer encoders with the ability to fuse information from multiple passages, using global representation to provide cross-sample attention over all tokens across samples. Furthermore, we propose an alternative answer span probability calculation to better aggregate answer scores in the global space of all samples. Using our proposed method, we outperform the current state-of-the-art method by 2.5 Exact Match score on the Natural Question dataset while using only 25% of parameters and 35% of the latency during inference, and 4.4 Exact Match on WebQuestions dataset. When coupled with synthetic data augmentation, we outperform larger models on the TriviaQA dataset as well. The latency and parameter savings of our method make it particularly attractive for open-domain question answering, as these models are often compute-intensive.
In Syntology View this paper on Syntology: its repositories, every harvested function with whether it ran, its licence and the call to fetch it.
Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Open-Domain Question Answering | WebQuestions | FiE+PAQ | Exact Match | 56.3 | #2 of 4 | Archive leaderboard | report |
| Open-Domain Question Answering | WebQuestions | FiE | Exact Match | 52.4 | #3 of 4 | Archive leaderboard | report |
| Question Answering | Natural Questions | FiE | EM | 58.4 | #4 of 47 | Archive leaderboard | report |
| Question Answering | TriviaQA | FiE+PAQ | EM | 72.6 | #24 of 56 | Archive leaderboard | report |
| Question Answering | WebQuestions | FiE+PAQ | EM | 56.3 | #5 of 37 | Archive leaderboard | report |
| Question Answering | WebQuestions | FiE | EM | 52.4 | #6 of 37 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections