Papers › AIC CTU system at AVeriTeC: Re-framing automated fact-checking as a simple RAG task

AIC CTU system at AVeriTeC: Re-framing automated fact-checking as a simple RAG task

15 Oct 2024arXiv:2410.11446archive 2025-07-28

Herbert Ullrich, Tomáš Mlynář, Jan Drchal

This paper describes our 3ʳᵈ place submission in the AVeriTeC shared task in which we attempted to address the challenge of fact-checking with evidence retrieved in the wild using a simple scheme of Retrieval-Augmented Generation (RAG) designed for the task, leveraging the predictive power of Large Language Models. We release our codebase and explain its two modules - the Retriever and the Evidence & Label generator - in detail, justifying their features such as MMR-reranking and Likert-scale confidence estimation. We evaluate our solution on AVeriTeC dev and test set and interpret the results, picking the GPT-4o as the most appropriate model for our pipeline at the time of our publication, with Llama 3.1 70B being a promising open-source alternative. We perform an empirical error analysis to see that faults in our predictions often coincide with noise in the data or ambiguous fact-checks, provoking further research and data augmentation.

Paper PDF Code

Code

aic-factcheck/aic_averitec officialmentioned in papermentioned on GitHubpytorchNOASSERTION report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Data Augmentation Fact Checking RAG Reranking Retrieval-augmented Generation

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Fact Checking	AVeriTeC	CTU AIC	AveriTeC	0.5	#2 of 3	Archive leaderboard	report
Fact Checking	AVeriTeC	CTU AIC	Question + Answer score	0.32	#2 of 3	Archive leaderboard	report
Fact Checking	AVeriTeC	CTU AIC	Question Only score	0.46	#2 of 3	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

LLaMA SET

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections