Browse State-of-the-Art › audio-visual event localization
audio-visual event localization
13 papers with code · 1 benchmark · 2 datasets archive 2025-07-28
Benchmarks archive 2025-07-28
1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| UnAV-100 (2 rows) | UnAV | Dense-Localizing Audio-Visual Events in Untrimmed Videos: A... | code | Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 8 where Syntology's instrument failed) · 2 unverified | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
2 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 26 · with a code link: 13 · where Syntology ran a sample: 10 (9 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (10 of 26 tagged: 9 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 26 of 26, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from.
-
1 Apr 2021 2 repositories listed Syntology official: harvested, nothing ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples) · 2 pointer-only (licence)To encourage the network to extract high correlated features for positive samples, a new audio-visual pair similarity loss is proposed.
-
20 Feb 2019 2 repositories listed Syntology 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples) · 6 pointer-only (licence)Audio-visual event localization requires one to identify theevent which is both visible and audible in a video (eitherat a frame or video level).
-
23 Mar 2018 2 repositories listed Syntology 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 4 pointer-only (licence)In this paper, we introduce a novel problem of audio-visual event localization in unconstrained videos.
-
17 Dec 2024 1 repository listed Syntology official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 10 where Syntology's instrument failed) · 1 unverified (of 14 harvested samples) · 14 pointer-only (licence)Specifically, the CMCC module contains two branches: a cross-modal interaction branch and a temporal consistency-gated branch.
-
18 Nov 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)In this paper, we advance the field by introducing the Open-Vocabulary Audio-Visual Event Localization (OV-AVEL) problem, which requires localizing audio-visual events and predicting explicit categories for both seen…
-
4 Aug 2024 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 9 harvested samples) · 1 pointer-only (licence)Specifically, we have enhanced the model's ability to discern subtle differences between event and background and improved the accuracy of event classification in our model.
-
4 Apr 2024 1 repository listed Syntology official (archive's flag): 13 ran · 13 ran (of which 1 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 9 where Syntology's instrument failed) · 2 unverified (of 15 harvested samples) · 8 pointer-only (licence)Video localization tasks aim to temporally locate specific instances in videos, including temporal action localization (TAL), sound event detection (SED) and audio-visual event localization (AVEL).
-
27 May 2023 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 2 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples)Audio-visual learning has been a major pillar of multi-modal machine learning, where the community mostly focused on its modality-aligned setting, i. e., the audio and visual modality are both assumed to signal the…
-
22 Mar 2023 1 repository listed Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 8 where Syntology's instrument failed) · 2 unverified (of 13 harvested samples) · 8 pointer-only (licence)To better adapt to real-life applications, in this paper we focus on the task of dense-localizing audio-visual events, which aims to jointly localize and recognize all audio-visual events occurring in an untrimmed video.
-
11 Oct 2022 1 repository listedIn contrast to existing methods, we propose a novel video-level semantic consistency guidance network for the AVE localization task.
-
16 Feb 2022 1 repository listedSelf-attention based Transformer models have demonstrated impressive results for image classification and object detection, and more recently for video understanding.
-
1 Jan 2022 1 repository listedAudiovisual Event (AVE) localization requires the model to jointly localize an event by observing audio and visual information.
-
24 Nov 2021 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples)Recognizing and localizing events in videos is a fundamental task for video understanding.
-
Audio-visual Event Localization on Portrait Mode Short Videos9 Apr 2025 0 repositories listedAudio-visual event localization (AVEL) plays a critical role in multimodal scene understanding.
-
Audio-Visual Semantic Graph Network for Audio-Visual Event Localization1 Jan 2025 0 repositories listedAudio-visual event localization (AVEL) aims to identify both the category and temporal boundaries of events that are both audible and visible in unconstrained videos.
-
Pilot-guided Multimodal Semantic Communication for Audio-Visual Event Localization9 Dec 2024 0 repositories listedMultimodal semantic communication, which integrates various data modalities such as text, images, and audio, significantly enhances communication efficiency and reliability.
-
Multimodal Trustworthy Semantic Communication for Audio-Visual Event Localization4 Nov 2024 0 repositories listedThis letter introduces MMTrustSC, a novel framework designed to address these challenges by enhancing the security and reliability of multimodal communication.
-
Label-anticipated Event Disentanglement for Audio-Visual Video Parsing11 Jul 2024 0 repositories listedAudio-Visual Video Parsing (AVVP) task aims to detect and temporally locate events within audio and visual modalities.
-
Advancing Weakly-Supervised Audio-Visual Video Parsing via Segment-wise Pseudo Labeling3 Jun 2024 0 repositories listedThe Audio-Visual Video Parsing task aims to identify and temporally localize the events that occur in either or both the audio and visual streams of audible videos.
-
Temporal Label-Refinement for Weakly-Supervised Audio-Visual Event Localization12 Jul 2023 0 repositories listedAudio-Visual Event Localization (AVEL) is the task of temporally localizing and classifying audio-visual events, i.e., events simultaneously visible and audible in a video.
-
AVE-CLIP: AudioCLIP-based Multi-window Temporal Transformer for Audio Visual Event Localization11 Oct 2022 0 repositories listedAn audio-visual event (AVE) is denoted by the correspondence of the visual and auditory signals in a video segment.
-
Past and Future Motion Guided Network for Audio Visual Event Localization8 May 2022 0 repositories listedIn recent years, audio-visual event localization has attracted much attention.
-
Multi-Modulation Network for Audio-Visual Event Localization26 Aug 2021 0 repositories listedWe study the problem of localizing audio-visual events that are both audible and visible in a video.
-
MPN: Multimodal Parallel Network for Audio-Visual Event Localization7 Apr 2021 0 repositories listedAudio-visual event localization aims to localize an event that is both audible and visible in the wild, which is a widespread audio-visual scene analysis task for unconstrained videos.
-
Audio-Visual Event Localization via Recursive Fusion by Joint Co-Attention14 Aug 2020 0 repositories listedThe major challenge in audio-visual event localization task lies in how to fuse information from multiple modalities effectively.
-
Dual Attention Matching for Audio-Visual Event Localization1 Oct 2019 0 repositories listedIn this paper, we investigate the audio-visual event localization problem.
Syntology lines on 10 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections