Datasets › NExT-QA

NExT-QA

Introduced by Junbin Xiao et al. in NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions19 Jun 2021 archive 2025-07-28

NExT-QA is a VideoQA benchmark targeting the explanation of video contents. It challenges QA models to reason about the causal and temporal actions and understand the rich object interactions in daily activities, e.g., "why is the boy crying?" and "How does the lady react after the boy fall backward?". It supports both multi-choice and generative open-ended QA tasks. The videos are untrimmed and the questions usually invoke local video contents for answers.

Benchmarks archive 2025-07-28

All 3 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

Syntology 2 newer papers were judged to report on this dataset's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 0 placed): Video Question Answering · NExT-QA (2 judged, 0 placed), Zero-Shot Video Question Answer · NExT-QA (2 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).

Papers archive 2025-07-28

papers with a benchmark row: 64 · with a code link: 52 · where Syntology ran a sample: 35 (29 with a run with no instrument failure, 6 where every run was a failure of Syntology's instrument) Syntology

Show: all papers with a benchmark rowonly where code ran (35 of 64 with a benchmark row: 29 with a run with no instrument failure, 6 where every run was a failure of Syntology's instrument)

Page 1 of 1: papers 1 to 64 of 64 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).

The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 174. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record.

PaperCodeResultsDateSamples run Syntology
VideoMultiAgents: A Multi-Agent Framework for Video Question Answering 1 1 25 Apr 2025 not harvested
PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding 1 3 17 Apr 2025 not harvested
Agentic Keyframe Search for Video Question Answering 1 1 20 Mar 2025 official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified
BIMBA: Selective-Scan Compression for Long-Range Video Question Answering 1 1 12 Mar 2025 not harvested
ENTER: Event Based Interpretable Reasoning for VideoQA 0 1 24 Jan 2025 not harvested
VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding 1 1 22 Jan 2025 official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 5 where Syntology's instrument failed) · 6 unverified (2 pointer-only for licence)
VidCtx: Context-aware Video Question Answering with Image Models 1 1 23 Dec 2024 not harvested
Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling 1 1 6 Dec 2024 official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 8 unverified
LinVT: Empower Your Image-level Large Language Model to Understand Videos 1 1 6 Dec 2024 official (archive's flag): 7 ran · 10 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 3 honoured, 1 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified (12 pointer-only for licence)
NVILA: Efficient Frontier Visual Language Models 2 1 5 Dec 2024 not harvested
TS-LLaVA: Constructing Visual Tokens through Thumbnail-and-Sampling for Training-Free Video Large Language Models 1 1 17 Nov 2024 official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 1 violated, 0 with no contract checked; 7 where Syntology's instrument failed) · 1 unverified (1 pointer-only for licence)
Video Instruction Tuning With Synthetic Data 0 1 3 Oct 2024 not harvested
Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolution 1 1 19 Sep 2024 official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 1 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (1 pointer-only for licence)
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution 8 1 18 Sep 2024 official (archive's flag): 4 ran · 12 ran (of which 1 constructed an object rather than computing a result; 10 with no instrument failure: 4 honoured, 1 violated, 5 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (1 pointer-only for licence)
LongVILA: Scaling Long-Context Visual Language Models for Long Videos 1 1 19 Aug 2024 not harvested
mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models 1 1 9 Aug 2024 not harvested
LLaVA-OneVision: Easy Visual Task Transfer 2 2 6 Aug 2024 not harvested
SlowFast-LLaVA: A Strong Training-Free Baseline for Video Large Language Models 1 1 22 Jul 2024 official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (6 pointer-only for licence)
LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models 3 3 10 Jul 2024 official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (4 pointer-only for licence)
Tarsier: Recipes for Training and Evaluating Large Video Description Models 1 1 30 Jun 2024 official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified
Long Context Transfer from Language to Vision 2 1 24 Jun 2024 official (archive's flag): 2 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (5 pointer-only for licence)
Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QA 1 1 13 Jun 2024 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (3 pointer-only for licence)
VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs 3 1 11 Jun 2024 community repositories only · 12 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 1 violated, 5 with no contract checked; 5 where Syntology's instrument failed) · 5 unverified (8 pointer-only for licence)
DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMs 0 1 6 Jun 2024 not harvested
VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos 1 1 29 May 2024 official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 2 honoured, 0 violated, 5 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified
MoReVQA: Exploring Modular Reasoning Models for Video Question Answering 0 1 9 Apr 2024 not harvested
TraveLER: A Modular Multi-LMM Agent Framework for Video Question-Answering 1 1 1 Apr 2024 not harvested
An Image Grid Can Be Worth a Video: Zero-shot Video Question Answering Using a VLM 1 2 27 Mar 2024 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified
Understanding Long Videos with Multimodal Language Models 1 1 25 Mar 2024 official (archive's flag): 3 ran · 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (1 pointer-only for licence)
Language Repository for Long Video Understanding 1 1 21 Mar 2024 official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 3 honoured, 0 violated, 0 with no contract checked; 6 where Syntology's instrument failed) · 0 unverified
VideoAgent: Long-form Video Understanding with Large Language Model as Agent 2 1 15 Mar 2024 not harvested
Efficient Temporal Extrapolation of Multimodal Large Language Models with Temporal Grounding Bridge 2 1 25 Feb 2024 official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (2 pointer-only for licence)
Question-Instructed Visual Descriptions for Zero-Shot Video Question Answering 1 1 16 Feb 2024 not harvested
CREMA: Generalizable and Efficient Video-Language Reasoning via Multimodal Modular Fusion 1 1 8 Feb 2024 official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 7 where Syntology's instrument failed) · 2 unverified (1 pointer-only for licence)
Glance and Focus: Memory Prompting for Multi-Event Video Question Answering 1 1 3 Jan 2024 official (archive's flag): 14 ran · 14 ran (of which 2 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 1 violated, 4 with no contract checked; 8 where Syntology's instrument failed) · 4 unverified (7 pointer-only for licence)
A Simple LLM Framework for Long-Range Video Question-Answering 1 2 28 Dec 2023 official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified
Gemini: A Family of Highly Capable Multimodal Models 1 2 19 Dec 2023 not harvested
Text-Conditioned Resampler For Long Form Video Understanding 0 1 19 Dec 2023 not harvested
ViLA: Efficient Video-Language Alignment for Video Question Answering 1 2 13 Dec 2023 official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified
RTQ: Rethinking Video-language Understanding Based on Image-text Model 2 1 1 Dec 2023 not harvested
Zero-Shot Video Question Answering with Procedural Programs 0 1 1 Dec 2023 not harvested
MVBench: A Comprehensive Multi-modal Video Understanding Benchmark 3 4 28 Nov 2023 official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 1 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (2 pointer-only for licence)
Vamos: Versatile Action Models for Video Understanding 1 1 22 Nov 2023 official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 3 honoured, 1 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (5 pointer-only for licence)
Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities 0 1 9 Nov 2023 not harvested
Large Language Models are Temporal and Causal Reasoners for Video Question Answering 1 1 24 Oct 2023 official: harvested, nothing ran · 0 ran · 4 unverified
PaLI-3 Vision Language Models: Smaller, Faster, Stronger 1 1 13 Oct 2023 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 3 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (1 pointer-only for licence)
Mistral 7B 6 1 10 Oct 2023 official (archive's flag): 9 ran · 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (2 pointer-only for licence)
ATM: Action Temporality Modeling for Video Question Answering 0 1 5 Sep 2023 not harvested
Emu: Generative Pretraining in Multimodality 2 1 11 Jul 2023 official: harvested for another paper · 0 ran · 2 unverified
Retrieving-to-Answer: Zero-Shot Video Question Answering with Frozen Large Language Models 0 1 15 Jun 2023 not harvested
PaLI-X: On Scaling up a Multilingual Vision and Language Model 2 1 29 May 2023 7 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 4 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (2 pointer-only for licence)
Paxion: Patching Action Knowledge in Video-Language Foundation Models 1 1 18 May 2023 official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (2 pointer-only for licence)
Self-Chained Image-Language Model for Video Localization and Question Answering 1 2 11 May 2023 official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (7 pointer-only for licence)
Verbs in Action: Improving verb understanding in video-language models 1 2 13 Apr 2023 official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified
ViperGPT: Visual Inference via Python Execution for Reasoning 1 2 14 Mar 2023 not harvested
Contrastive Video Question Answering via Video Graph Transformer 1 2 27 Feb 2023 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified
Semi-Parametric Video-Grounded Text Generation 0 1 27 Jan 2023 not harvested
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training 0 1 30 Dec 2022 not harvested
MIST: Multi-modal Iterative Spatial-Temporal Transformer for Long-form Video Question Answering 1 1 19 Dec 2022 not harvested
Video Graph Transformer for Video Question Answering 1 2 12 Jul 2022 official (archive's flag): 9 ran · 9 ran (of which 5 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified
Revisiting the "Video" in Video-Language Understanding 1 1 3 Jun 2022 3 ran (of which 2 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified
Flamingo: a Visual Language Model for Few-Shot Learning 5 2 29 Apr 2022 18 ran (of which 6 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 1 violated, 10 with no contract checked; 7 where Syntology's instrument failed) · 6 unverified (8 pointer-only for licence)
(2.5+1)D Spatio-Temporal Scene Graphs for Video Question Answering 0 1 18 Feb 2022 not harvested
Video as Conditional Graph Hierarchy for Multi-Granular Question Answering 1 1 12 Dec 2021 official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (1 pointer-only for licence)

Dataset loaders archive 2025-07-28

No loader listed in the archive.

Tasks archive 2025-07-28

License archive 2025-07-28

MIT

Modalities archive 2025-07-28

Languages archive 2025-07-28

Variants archive 2025-07-28

  • NExT-QA
  • NExT-GQA
  • NExT-QA (Open-ended VideoQA)

3 variant names, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections