Papers › Longformer for MS MARCO Document Re-ranking Task

Longformer for MS MARCO Document Re-ranking Task

20 Sep 2020arXiv:2009.09392archive 2025-07-28

Ivan Sekulić, Amir Soleimani, Mohammad Aliannejadi, Fabio Crestani

Two step document ranking, where the initial retrieval is done by a classical information retrieval method, followed by neural re-ranking model, is the new standard. The best performance is achieved by using transformer-based models as re-rankers, e.g., BERT. We employ Longformer, a BERT-like model for long documents, on the MS MARCO document re-ranking task. The complete code used for training the model can be found on: https://github.com/isekulic/longformer-marco

Paper PDF Code

Code

isekulic/longformer-marco officialmentioned in papermentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Document Ranking Information Retrieval Re-Ranking Retrieval

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

Adam AdamW Attention Attention Dropout BERT Dense Connections Dropout Layer Normalization Linear Layer Linear Warmup With Linear Decay Longformer Multi-Head Attention Residual Connection Softmax Weight Decay WordPiece

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections