Active Speaker Detection

Benchmarks archive 2025-07-28

1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

2 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

1 subtask in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 30 papers with code (63 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection
  14 Jul 2021 4 repositories listed Syntology ran 2 of 9 samples · 7 unverified · 1 pointer-only (licence)
  Active speaker detection (ASD) seeks to detect who is speaking in a visual scene of one or more speakers.
• End-to-End Active Speaker Detection
  27 Mar 2022 3 repositories listed Syntology ran 3 of 5 samples · 2 unverified · 5 pointer-only (licence)
  Recent advances in the Active Speaker Detection (ASD) problem build upon a two-stage process: feature extraction and spatio-temporal context aggregation.
• LoCoNet: Long-Short Context Network for Active Speaker Detection
  19 Jan 2023 2 repositories listed
  These two contexts are complementary to each other and can help infer the active speaker.
• Learning Long-Term Spatial-Temporal Graphs for Active Speaker Detection
  15 Jul 2022 2 repositories listed
  Active speaker detection (ASD) in videos with multiple speakers is a challenging task as it requires learning effective audiovisual features and spatial-temporal correlations over long temporal windows.
• AVA-ActiveSpeaker: An Audio-Visual Dataset for Active Speaker Detection
  5 Jan 2019 2 repositories listed
  The dataset contains temporally labeled face tracks in video, where each face instance is labeled as speaking or not, and whether the speech is audible.
• UniTalk: Towards Universal Active Speaker Detection in Real World Scenarios
  28 May 2025 1 repository listed
  We present UniTalk, a novel dataset specifically designed for the task of active speaker detection, emphasizing challenging scenarios to enhance model generalization.
• CoGenAV: Versatile Audio-Visual Representation Learning via Contrastive-Generative Synchronization
  6 May 2025 1 repository listed
  They also enable strong performance in Visual Speech Recognition (VSR) with a WER of 20.
• LASER: Lip Landmark Assisted Speaker Detection for Robustness
  21 Jan 2025 1 repository listed
  Active Speaker Detection (ASD) aims to identify speaking individuals in complex visual scenes.
• ASDnB: Merging Face with Body Cues For Robust Active Speaker Detection
  11 Dec 2024 1 repository listed
  State-of-the-art Active Speaker Detection (ASD) approaches mainly use audio and facial features as input.
• BIAS: A Body-based Interpretable Active Speaker Approach
  6 Dec 2024 1 repository listed
  The results show that BIAS is state-of-the-art in challenging conditions where body-based features are of utmost importance (Columbia, open-settings, and WASD), and yields competitive results in AVA-ActiveSpeaker, where…
• FabuLight-ASD: Unveiling Speech Activity via Body Language
  20 Nov 2024 1 repository listed
  Active speaker detection (ASD) in multimodal environments is crucial for various applications, from video conferencing to human-robot interaction.
• Imitation of human motion achieves natural head movements for humanoid robots in an active-speaker detection task
  16 Jul 2024 1 repository listed
  Head movements are crucial for social human-human interaction.
• AnnoTheia: A Semi-Automatic Annotation Toolkit for Audio-Visual Speech Technologies
  20 Feb 2024 1 repository listed
  In order to promote research on low-resource languages for audio-visual speech technologies, we present AnnoTheia, a semi-automatic annotation toolkit that detects when a person speaks on the scene and the corresponding…
• Leveraging Visual Supervision for Array-based Active Speaker Detection and Localization
  21 Dec 2023 1 repository listed
  The multichannel audio ``student'' network is trained to generate the same results.
• GestSync: Determining who is speaking without a talking head
  8 Oct 2023 1 repository listed
  In this paper we introduce a new synchronisation task, Gesture-Sync: determining if a person's gestures are correlated with their speech or not.
• TalkNCE: Improving Active Speaker Detection with Talk-Aware Contrastive Learning
  21 Sep 2023 1 repository listed
  The goal of this work is Active Speaker Detection (ASD), a task to determine whether a person is speaking or not in a series of video frames.
• Target Active Speaker Detection with Audio-visual Cues
  22 May 2023 1 repository listed
  To benefit from both facial cue and reference speech, we propose the Target Speaker TalkNet (TS-TalkNet), which leverages a pre-enrolled speaker embedding to complement the audio-visual synchronization cue in detecting…
• WASD: A Wilder Active Speaker Detection Dataset
  9 Mar 2023 1 repository listed
  The results show that: 1) AVA trained models maintain a state-of-the-art performance in WASD Easy group, while underperforming in the Hard one, showing the 2) similarity between AVA and Easy data; and 3) training in…
• A Light Weight Model for Active Speaker Detection
  8 Mar 2023 1 repository listed Syntology ran 5 of 6 samples · 1 unverified
  Experimental results on the AVA-ActiveSpeaker dataset show that our framework achieves competitive mAP performance (94.
• Audio-Visual Activity Guided Cross-Modal Identity Association for Active Speaker Detection
  1 Dec 2022 1 repository listed
  Active speaker detection in videos addresses associating a source face, visible in the video frames, with the underlying speech in the audio modality.
• Whose Emotion Matters? Speaking Activity Localisation without Prior Knowledge
  23 Nov 2022 1 repository listed
  Finally, we devise a model for emotion recognition in conversations trained on the realigned MELD-FAIR videos, which outperforms state-of-the-art models for ERC based on vision alone.
• Unsupervised active speaker detection in media content using cross-modal information
  24 Sep 2022 1 repository listed
  We leverage speaker identity information from speech and faces, and formulate active speaker detection as a speech-face assignment task such that the active speaker's face and the underlying speech identify the same…
• Look\&Listen: Multi-Modal Correlation Learning for Active Speaker Detection and Speech Enhancement
  4 Mar 2022 1 repository listed
  Active speaker detection and speech enhancement have become two increasingly attractive topics in audio-visual scenario understanding.
• Look Who's Talking: Active Speaker Detection in the Wild
  17 Aug 2021 1 repository listed
  Face tracks are extracted from the videos and active segments are annotated based on the timestamps of VoxConverse in a semi-automatic way.
• How to Design a Three-Stage Architecture for Audio-Visual Active Speaker Detection in the Wild
  7 Jun 2021 1 repository listed
  Successful active speaker detection requires a three-stage pipeline: (i) audio-visual encoding for all speakers in the clip, (ii) inter-speaker relation modeling between a reference speaker and the background speakers…
• NUS-HLT Report for ActivityNet Challenge 2021 AVA (Speaker)
  1 Jun 2021 1 repository listed
  Active speaker detection (ASD) seeks to detect who is speaking in a visual scene of one or more speakers.
• MAAS: Multi-modal Assignation for Active Speaker Detection
  11 Jan 2021 1 repository listed
  Active speaker detection requires a solid integration of multi-modal cues.
• Self-Supervised Learning of Audio-Visual Objects from Video
  10 Aug 2020 1 repository listed Syntology ran 1 of 14 samples · 13 unverified
  Our objective is to transform a video into a set of discrete audio-visual objects using self-supervised learning.
• Active Speakers in Context
  20 May 2020 1 repository listed
  Current methods for active speak er detection focus on modeling short-term audiovisual information from a single speaker.
• Bio-Inspired Modality Fusion for Active Speaker Detection
  28 Feb 2020 1 repository listed
  Furthermore, neuroscience has successfully identified the superior colliculus region in the brain as the one responsible for this modality fusion, with a handful of biological models having been proposed to approach its…

Syntology lines on 4 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.