Browse State-of-the-Art › Active Speaker Detection
Active Speaker Detection
30 papers with code · 1 benchmark · 2 datasets archive 2025-07-28
Benchmarks archive 2025-07-28
1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| LRS3-TED (1 row) | GestSync | GestSync: Determining who is speaking without a talking head | code | — | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
2 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
1 subtask in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 63 · with a code link: 30 · where Syntology ran a sample: 4 (4 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (4 of 63 tagged: 4 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 63 of 63, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection14 Jul 2021 4 repositories listed Syntology official (archive's flag): 5 ran · 7 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 9 harvested samples) · 1 pointer-only (licence)Active speaker detection (ASD) seeks to detect who is speaking in a visual scene of one or more speakers.
-
27 Mar 2022 3 repositories listed Syntology official (archive's flag): 2 ran · 3 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 5 harvested samples) · 5 pointer-only (licence)Recent advances in the Active Speaker Detection (ASD) problem build upon a two-stage process: feature extraction and spatio-temporal context aggregation.
-
19 Jan 2023 2 repositories listedThese two contexts are complementary to each other and can help infer the active speaker.
-
15 Jul 2022 2 repositories listedActive speaker detection (ASD) in videos with multiple speakers is a challenging task as it requires learning effective audiovisual features and spatial-temporal correlations over long temporal windows.
-
5 Jan 2019 2 repositories listedThe dataset contains temporally labeled face tracks in video, where each face instance is labeled as speaking or not, and whether the speech is audible.
-
28 May 2025 1 repository listedWe present UniTalk, a novel dataset specifically designed for the task of active speaker detection, emphasizing challenging scenarios to enhance model generalization.
-
6 May 2025 1 repository listedThey also enable strong performance in Visual Speech Recognition (VSR) with a WER of 20.
-
21 Jan 2025 1 repository listedActive Speaker Detection (ASD) aims to identify speaking individuals in complex visual scenes.
-
11 Dec 2024 1 repository listedState-of-the-art Active Speaker Detection (ASD) approaches mainly use audio and facial features as input.
-
6 Dec 2024 1 repository listedThe results show that BIAS is state-of-the-art in challenging conditions where body-based features are of utmost importance (Columbia, open-settings, and WASD), and yields competitive results in AVA-ActiveSpeaker, where…
-
20 Nov 2024 1 repository listedActive speaker detection (ASD) in multimodal environments is crucial for various applications, from video conferencing to human-robot interaction.
-
16 Jul 2024 1 repository listedHead movements are crucial for social human-human interaction.
-
20 Feb 2024 1 repository listedIn order to promote research on low-resource languages for audio-visual speech technologies, we present AnnoTheia, a semi-automatic annotation toolkit that detects when a person speaks on the scene and the corresponding…
-
21 Dec 2023 1 repository listedThe multichannel audio ``student'' network is trained to generate the same results.
-
8 Oct 2023 1 repository listedIn this paper we introduce a new synchronisation task, Gesture-Sync: determining if a person's gestures are correlated with their speech or not.
-
21 Sep 2023 1 repository listedThe goal of this work is Active Speaker Detection (ASD), a task to determine whether a person is speaking or not in a series of video frames.
-
22 May 2023 1 repository listedTo benefit from both facial cue and reference speech, we propose the Target Speaker TalkNet (TS-TalkNet), which leverages a pre-enrolled speaker embedding to complement the audio-visual synchronization cue in detecting…
-
9 Mar 2023 1 repository listedThe results show that: 1) AVA trained models maintain a state-of-the-art performance in WASD Easy group, while underperforming in the Hard one, showing the 2) similarity between AVA and Easy data; and 3) training in…
-
8 Mar 2023 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples)Experimental results on the AVA-ActiveSpeaker dataset show that our framework achieves competitive mAP performance (94.
-
1 Dec 2022 1 repository listedActive speaker detection in videos addresses associating a source face, visible in the video frames, with the underlying speech in the audio modality.
-
23 Nov 2022 1 repository listedFinally, we devise a model for emotion recognition in conversations trained on the realigned MELD-FAIR videos, which outperforms state-of-the-art models for ERC based on vision alone.
-
24 Sep 2022 1 repository listedWe leverage speaker identity information from speech and faces, and formulate active speaker detection as a speech-face assignment task such that the active speaker's face and the underlying speech identify the same…
-
4 Mar 2022 1 repository listedActive speaker detection and speech enhancement have become two increasingly attractive topics in audio-visual scenario understanding.
-
17 Aug 2021 1 repository listedFace tracks are extracted from the videos and active segments are annotated based on the timestamps of VoxConverse in a semi-automatic way.
-
7 Jun 2021 1 repository listedSuccessful active speaker detection requires a three-stage pipeline: (i) audio-visual encoding for all speakers in the clip, (ii) inter-speaker relation modeling between a reference speaker and the background speakers…
-
1 Jun 2021 1 repository listedActive speaker detection (ASD) seeks to detect who is speaking in a visual scene of one or more speakers.
-
11 Jan 2021 1 repository listedActive speaker detection requires a solid integration of multi-modal cues.
-
10 Aug 2020 1 repository listed Syntology 11 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 0 violated, 11 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 14 harvested samples)Our objective is to transform a video into a set of discrete audio-visual objects using self-supervised learning.
-
20 May 2020 1 repository listedCurrent methods for active speak er detection focus on modeling short-term audiovisual information from a single speaker.
-
28 Feb 2020 1 repository listedFurthermore, neuroscience has successfully identified the superior colliculus region in the brain as the one responsible for this modality fusion, with a handful of biological models having been proposed to approach its…
-
Understanding Co-speech Gestures in-the-wild28 Mar 2025 0 repositories listed
-
How to Squeeze An Explanation Out of Your Model6 Dec 2024 0 repositories listed
-
An Efficient and Streaming Audio Visual Active Speaker Detection System13 Sep 2024 0 repositories listed
-
Audio-Visual Talker Localization in Video for Spatial Sound Reproduction1 Jun 2024 0 repositories listed
-
Enhancing Real-World Active Speaker Detection with Multi-Modal Extraction Pre-Training1 Apr 2024 0 repositories listed
-
Robust Active Speaker Detection in Noisy Environments27 Mar 2024 0 repositories listed
-
Audio-visual child-adult speaker classification in dyadic interactions3 Oct 2023 0 repositories listed
-
A Real-Time Active Speaker Detection System Integrating an Audio-Visual Signal with a Spatial Querying Mechanism15 Sep 2023 0 repositories listed
-
Audio Inputs for Active Speaker Detection and Localization via Microphone Array27 Jul 2023 0 repositories listed
-
Learning Spatial Features from Audio-Visual Correspondence in Egocentric Videos10 Jul 2023 0 repositories listed
-
Deep Learning Based Audio-Visual Multi-Speaker DOA Estimation Using Permutation-Free Loss Function26 Oct 2022 0 repositories listed
-
Intel Labs at Ego4D Challenge 2022: A Better Baseline for Audio-Visual Diarization14 Oct 2022 0 repositories listed
-
Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detection3 Oct 2022 0 repositories listed
-
End-To-End Audiovisual Feature Fusion for Active Speaker Detection27 Jul 2022 0 repositories listed
-
22 Jun 2022 0 repositories listed
-
Rethinking Audio-visual Synchronization for Active Speaker Detection21 Jun 2022 0 repositories listed
-
Audio-video fusion strategies for active speaker detection in meetings9 Jun 2022 0 repositories listed
-
Best of Both Worlds: Multi-task Audio-Visual Automatic Speech Recognition and Active Speaker Detection10 May 2022 0 repositories listed
-
Visually Supervised Speaker Detection and Localization via Microphone Array7 Mar 2022 0 repositories listed
-
Data standardization for robust lip sync13 Feb 2022 0 repositories listed
-
6 Jan 2022 0 repositories listed
-
Learning Spatial-Temporal Graphs for Active Speaker Detection2 Dec 2021 0 repositories listed
-
FaVoA: Face-Voice Association Favours Ambiguous Speaker Detection1 Sep 2021 0 repositories listed
-
5 Aug 2021 0 repositories listed
-
7 Jun 2021 0 repositories listed
-
1 Jun 2021 0 repositories listed
-
Spot the conversation: speaker diarisation in the wild2 Jul 2020 0 repositories listed
-
Multimodal active speaker detection and virtual cinematography for video conferencing10 Feb 2020 0 repositories listed
-
25 Jun 2019 0 repositories listed
-
1 Jun 2019 0 repositories listed
-
Detection and Analysis of Content Creator Collaborations in YouTube Videos using Face- and Speaker-Recognition5 Jul 2018 0 repositories listed
-
Self-Supervised Vision-Based Detection of the Active Speaker as Support for Socially-Aware Language Acquisition24 Nov 2017 0 repositories listed
-
Cross-modal Supervision for Learning Active Speaker Detection in Video29 Mar 2016 0 repositories listed
Syntology lines on 4 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections