Papers › Improving Conditioning in Context-Aware Sequence to Sequence Models
Improving Conditioning in Context-Aware Sequence to Sequence Models
Xinyi Wang, Jason Weston, Michael Auli, Yacine Jernite
Neural sequence to sequence models are well established for applications which can be cast as mapping a single input sequence into a single output sequence. In this work, we focus on cases where generation is conditioned on both a short query and a long context, such as abstractive question answering or document-level translation. We modify the standard sequence-to-sequence approach to make better use of both the query and the context by expanding the conditioning mechanism to intertwine query and context attention. We also introduce a simple and efficient data augmentation method for the proposed model. Experiments on three different tasks show that both changes lead to consistent improvements.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
1 archive task tag without a task page not shown.
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Open-Domain Question Answering | ELI5 | Multi-Inrerleave | Rouge-1 | 23.32 | #6 of 6 | Archive leaderboard | report |
| Open-Domain Question Answering | ELI5 | Multi-Inrerleave | Rouge-2 | 4.79 | #6 of 6 | Archive leaderboard | report |
| Open-Domain Question Answering | ELI5 | Multi-Inrerleave | Rouge-L | 14.63 | #6 of 6 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections