Audio-visual Question Answering

Benchmarks archive 2025-07-28

1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

2 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

1 subtask in the archive's task tree.

Most implemented papers archive 2025-07-28

19 shown of 19 papers with code (27 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset
  29 May 2023 2 repositories listed Syntology ran 15 of 42 samples · 27 unverified
  Based on the proposed VAST-27M dataset, we train an omni-modality video-text foundational model named VAST, which can perceive and process vision, audio, and subtitle modalities from video, and better support various…
• FortisAVQA and MAVEN: a Benchmark Dataset and Debiasing Framework for Robust Multimodal Reasoning
  1 Apr 2025 1 repository listed
  The first stage expands the test space with greater diversity, while the second enables a refined robustness evaluation across rare, frequent, and overall question distributions.
• PAVE: Patching and Adapting Video Large Language Models
  25 Mar 2025 1 repository listed
  Pre-trained video large language models (Video LLMs) exhibit remarkable reasoning capabilities, yet adapting these models to new tasks involving additional modalities or data types (e.
• Question-Aware Gaussian Experts for Audio-Visual Question Answering
  6 Mar 2025 1 repository listed
  However, existing methods mainly use question information implicitly, limiting focus on question-specific details.
• AVQACL: A Novel Benchmark for Audio-Visual Question Answering Continual Learning
  1 Jan 2025 1 repository listed
  In this paper, a novel benchmark for audio-visual question answering continual learning (AVQACL) is introduced, aiming to study fine-grained scene understanding and spatial-temporal reasoning in videos under a continual…
• Boosting Audio Visual Question Answering via Key Semantic-Aware Cues
  30 Jul 2024 1 repository listed Syntology ran 2 of 3 samples · 1 unverified · 3 pointer-only (licence)
  The Audio Visual Question Answering (AVQA) task aims to answer questions related to various visual objects, sounds, and their interactions in videos.
• Learning Trimodal Relation for AVQA with Missing Modality
  23 Jul 2024 1 repository listed Syntology ran 20 of 24 samples · 4 unverified · 3 pointer-only (licence)
  Recent Audio-Visual Question Answering (AVQA) methods rely on complete visual and audio input to answer questions accurately.
• Towards Multilingual Audio-Visual Question Answering
  13 Jun 2024 1 repository listed
  In this paper, we work towards extending Audio-Visual Question Answering (AVQA) to multilingual settings.
• Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answering
  18 Apr 2024 1 repository listed Syntology ran 2 of 3 samples · 1 unverified · 1 pointer-only (licence)
  The former leads to a large, diverse test space, while the latter results in a comprehensive robustness evaluation on rare, frequent, and overall questions.
• Answering Diverse Questions via Text Attached with Key Audio-Visual Clues
  11 Mar 2024 1 repository listed
  Audio-visual question answering (AVQA) requires reference to video content and auditory information, followed by correlating the question to predict the most precise answer.
• CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarios
  7 Mar 2024 1 repository listed Syntology ran 6 of 12 samples · 6 unverified
  This paper focuses on the challenge of answering questions in scenarios that are composed of rich and complex dynamic audio-visual components.
• Object-aware Adaptive-Positivity Learning for Audio-Visual Question Answering
  20 Dec 2023 1 repository listed Syntology ran 0 of 1 samples · 1 unverified · 1 pointer-only (licence)
  These selected pairs are constrained to have larger similarity values than the mismatched pairs.
• Progressive Spatio-temporal Perception for Audio-Visual Question Answering
  10 Aug 2023 1 repository listed
  Such naturally multi-modal videos are composed of rich and complex dynamic audio-visual components, where most of which could be unrelated to the given questions, or even play as interference in answering the content of…
• Target-Aware Spatio-Temporal Reasoning via Answering Questions in Dynamics Audio-Visual Scenarios
  21 May 2023 1 repository listed
  Recent works rely on elaborate target-agnostic parsing of audio-visual scenes for spatial grounding while mistreating audio and video as separate entities for temporal grounding.
• VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset
  17 Apr 2023 1 repository listed
  Different from widely-studied vision-language pretraining models, VALOR jointly models relationships of vision, audio and language in an end-to-end manner.
• Vision Transformers are Parameter-Efficient Audio-Visual Learners
  15 Dec 2022 1 repository listed Syntology ran 1 of 2 samples · 1 unverified · 2 pointer-only (licence)
  To do so, we propose a latent audio-visual hybrid (LAVISH) adapter that adapts pretrained ViTs to audio-visual tasks by injecting a small number of trainable parameters into every layer of a frozen ViT.
• Learning to Answer Questions in Dynamic Audio-Visual Scenarios
  26 Mar 2022 1 repository listed Syntology ran 0 of 4 samples · 4 unverified
  In this paper, we focus on the Audio-Visual Question Answering (AVQA) task, which aims to answer questions regarding different visual objects, sounds, and their associations in videos.
• Pano-AVQA: Grounded Audio-Visual Question Answering on 360° Videos
  11 Oct 2021 1 repository listed
  However, previous benchmark tasks for panoramic videos are still limited to evaluate the semantic understanding of audio-visual relationships or spherical spatial property in surroundings.
• Pano-AVQA: Grounded Audio-Visual Question Answering on 360deg Videos
  1 Jan 2021 1 repository listed
  However, previous benchmark tasks for panoramic videos are still limited to evaluate the semantic understanding of audio-visual relationships or spherical spatial property in surroundings.

Syntology lines on 8 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.