TrecQA (Text Retrieval Conference Question Answering)

Introduced in What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA1 Jan 2007 archive 2025-07-28

Text Retrieval Conference Question Answering (TrecQA) is a dataset created from the TREC-8 (1999) to TREC-13 (2004) Question Answering tracks. There are two versions of TrecQA: raw and clean. Both versions have the same training set but their development and test sets differ. The commonly used clean version of the dataset excludes questions in development and test sets with no answers or only positive/negative answers. The clean version has 1,229/65/68 questions and 53,417/1,117/1,442 question-answer pairs for the train/dev/test split.

Source: A Gated Self-attention Memory Network for Answer Selection

Homepage Benchmarks Papers JSON

Benchmarks archive 2025-07-28

All 3 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

		First row (archive order)	Paper	Code
Question Answering	TrecQA	TANDA DeBERTa-V3-Large + ALL MAP 0.954	Structural Self-Supervised Objectives for Transformers	lucadiliello/transformers-framework	13	Compare
Answer Selection	TrecQA	RLAS-BIABC MAP 0.913	RLAS-BIABC: A Reinforcement Learning-Based Answer...	—	1	Compare
Text Classification	trec	no rows	—	—	0	Compare

Papers archive 2025-07-28

12 shown of 12 papers with a leaderboard row on this dataset's benchmarks, newest first. The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset; the archive's count for this dataset is 73. The Syntology column is from Syntology's graph (read 2026-09-24), stated per sample; it is not part of any archive number.

			Date	Samples run Syntology
Structural Self-Supervised Objectives for Transformers	1	1	15 Sep 2023	not harvested
Context-Aware Transformer Pre-Training for Answer Sentence Selection	0	1	24 May 2023	not harvested
RLAS-BIABC: A Reinforcement Learning-Based Answer Selection Using the BERT Model Boosted by an Improved ABC Algorithm	0	2	7 Jan 2023	not harvested
Pre-training Transformer Models with Sentence-Level Objectives for Answer Sentence Selection	0	2	20 May 2022	not harvested
Paragraph-based Transformer Pre-training for Multi-Sentence Inference	1	1	2 May 2022	not harvested
TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection	2	1	11 Nov 2019	not harvested
Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications	5	1	6 Jun 2019	not harvested
A Compare-Aggregate Model with Latent Clustering for Answer Selection	0	1	30 May 2019	not harvested
aNMM: Ranking Short Answer Texts with Attention-Based Neural Matching Model	1	1	5 Jan 2018	not harvested
Hyperbolic Representation Learning for Fast and Efficient Neural Question Answering	1	1	25 Jul 2017	not harvested
Pairwise Word Interaction Modeling with Deep Neural Networks for Semantic Similarity Measurement	0	1	1 Jun 2016	not harvested
Deep Learning for Answer Sentence Selection	2	1	4 Dec 2014	not harvested

Dataset loaders archive 2025-07-28

huggingface.co/datasets/CogComp/trec huggingface/datasetstfpytorchjax

huggingface.co/datasets/trec huggingface/datasetstfpytorchjax

2 loaders as listed in the archive; links are outbound and not re-checked here.

Tasks archive 2025-07-28

Question Answering Text Classification Answer Selection

License archive 2025-07-28

Unknown

Modalities archive 2025-07-28

Texts

Languages archive 2025-07-28

No language tagged.

Variants archive 2025-07-28

TrecQA
trec

2 variant names, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections