Audio Classification

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

25 leaderboard tables shown for this task, 22 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted. 10 shown of 25 until expanded.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

43 datasets whose archive record lists this task, ordered by the archive's paper count. 30 shown of 43 until expanded.

Subtasks archive 2025-07-28

4 subtasks in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 183 papers with code (361 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• CNN Architectures for Large-Scale Audio Classification
  29 Sep 2016 16 repositories listed Syntology ran 0 of 3 samples · 3 unverified
  Convolutional Neural Networks (CNNs) have proven very effective in image classification and show promise for audio.
• Perceiver: General Perception with Iterative Attention
  4 Mar 2021 12 repositories listed Syntology ran 41 of 55 samples · 14 unverified · 11 pointer-only (licence)
  The perception models used in deep learning on the other hand are designed for individual modalities, often relying on domain-specific assumptions such as the local grid structures exploited by virtually all existing…
• LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment
  3 Oct 2023 6 repositories listed Syntology ran 7 of 14 samples · 7 unverified
  We thus propose VIDAL-10M with Video, Infrared, Depth, Audio and their corresponding Language, naming as VIDAL-10M.
• VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text
  22 Apr 2021 5 repositories listed Syntology ran 5 of 8 samples · 3 unverified · 8 pointer-only (licence)
  We train VATT end-to-end from scratch using multimodal contrastive losses and evaluate its performance by the downstream tasks of video action recognition, audio event classification, image classification, and…
• AST: Audio Spectrogram Transformer
  5 Apr 2021 5 repositories listed Syntology ran 0 of 3 samples · 3 unverified
  In the past decade, convolutional neural networks (CNNs) have been widely adopted as the main building block for end-to-end audio classification models, which aim to learn a direct mapping from audio spectrograms to…
• Multi-level Attention Model for Weakly Supervised Audio Classification
  6 Mar 2018 5 repositories listed Syntology ran 0 of 11 samples · 11 unverified
  The objective of audio classification is to predict the presence or absence of audio events in an audio clip.
• BEATs: Audio Pre-Training with Acoustic Tokenizers
  18 Dec 2022 4 repositories listed Syntology ran 3 of 18 samples · 15 unverified
  In the first iteration, we use random projection as the acoustic tokenizer to train an audio SSL model in a mask and label prediction manner.
• Masked Autoencoders that Listen
  13 Jul 2022 4 repositories listed Syntology ran 17 of 32 samples · 15 unverified · 10 pointer-only (licence)
  Following the Transformer encoder-decoder design in MAE, our Audio-MAE first encodes audio spectrogram patches with a high masking ratio, feeding only the non-masked tokens through encoder layers.
• ATST: Audio Representation Learning with Teacher-Student Transformer
  26 Apr 2022 4 repositories listed
  Self-supervised learning (SSL) learns knowledge from a large amount of unlabeled data, and then transfers the knowledge to a specific problem with a limited number of labeled data.
• LEAF: A Learnable Frontend for Audio Classification
  21 Jan 2021 4 repositories listed Syntology ran 14 of 26 samples · 12 unverified · 2 pointer-only (licence)
  In this work we show that we can train a single learnable frontend that outperforms mel-filterbanks on a wide range of audio signals, including speech, music, audio events and animal sounds, providing a general-purpose…
• AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights
  15 Jun 2020 4 repositories listed Syntology ran 4 of 4 samples · 0 unverified
  Because of the scale invariance, this modification only alters the effective step sizes without changing the effective update directions, thus enjoying the original convergence properties of GD optimizers.
• Leveraging tropical reef, bird and unrelated sounds for superior transfer learning in marine bioacoustics
  25 Apr 2024 3 repositories listed
  Machine learning has the potential to revolutionize passive acoustic monitoring (PAM) for ecological assessments.
• SSAST: Self-Supervised Audio Spectrogram Transformer
  19 Oct 2021 3 repositories listed Syntology ran 11 of 16 samples · 5 unverified
  However, pure Transformer models tend to require more training data compared to CNNs, and the success of the AST relies on supervised pretraining that requires a large amount of labeled data and a complex training…
• Rethinking CNN Models for Audio Classification
  22 Jul 2020 3 repositories listed
  Besides, we show that even though we use the pretrained model weights for initialization, there is variance in performance in various output runs of the same model.
• Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition
  28 May 2019 3 repositories listed Syntology ran 0 of 13 samples · 13 unverified
  Modern deep neural networks are well known to be brittle in the face of unknown data instances and recognition of the latter remains a challenge.
• AudioMNIST: Exploring Explainable Artificial Intelligence for Audio Analysis on a Simple Benchmark
  9 Jul 2018 3 repositories listed Syntology ran 2 of 2 samples · 0 unverified
  Explainable Artificial Intelligence (XAI) is targeted at understanding how models perform feature selection and derive their classification decisions.
• Convolutional RNN: an Enhanced Model for Extracting Features from Sequential Data
  18 Feb 2016 3 repositories listed Syntology ran 0 of 2 samples · 2 unverified
  Traditional convolutional layers extract features from patches of data by applying a non-linearity on an affine function of the input.
• M2D2: Exploring General-purpose Audio-Language Representations Beyond CLAP
  28 Mar 2025 2 repositories listed
  In the second stage, it learns CLAP features using the audio features learned from the LLM-based embeddings.
• M2D-CLAP: Masked Modeling Duo Meets CLAP for Learning General-purpose Audio-Language Representation
  4 Jun 2024 2 repositories listed
  Here, we explore a new representation, a general-purpose audio-language representation, that performs well in both ZS and transfer learning.
• Masked Modeling Duo: Towards a Universal Audio Pre-training Framework
  9 Apr 2024 2 repositories listed Syntology ran 7 of 7 samples · 0 unverified · 7 pointer-only (licence)
  This study proposes Masked Modeling Duo (M2D), an improved masked prediction SSL, which learns by predicting representations of masked input signals that serve as training signals.
• InternVideo2: Scaling Foundation Models for Multimodal Video Understanding
  22 Mar 2024 2 repositories listed
  We introduce InternVideo2, a new family of video foundation models (ViFM) that achieve the state-of-the-art results in video recognition, video-text tasks, and video-centric dialogue.
• Acoustic Prompt Tuning: Empowering Large Language Models with Audition Capabilities
  30 Nov 2023 2 repositories listed
  In this work, we introduce Acoustic Prompt Tuning (APT), a new adapter extending LLMs and VLMs to the audio domain by injecting audio embeddings to the input of LLMs, namely soft prompting.
• Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models
  14 Nov 2023 2 repositories listed Syntology ran 5 of 7 samples · 2 unverified · 7 pointer-only (licence)
  Recently, instruction-following audio-language models have received broad attention for audio interaction with humans.
• Audio classification with Dilated Convolution with Learnable Spacings
  25 Sep 2023 2 repositories listed
  Dilated convolution with learnable spacings (DCLS) is a recent convolution method in which the positions of the kernel elements are learned throughout training by backpropagation.
• Global birdsong embeddings enable superior transfer learning for bioacoustic classification
  12 Jul 2023 2 repositories listed Syntology ran 4 of 7 samples · 3 unverified
  With the advent of deep learning models, classification of important signals from these datasets has markedly improved.
• Self-supervised Audio Teacher-Student Transformer for Both Clip-level and Frame-level Tasks
  7 Jun 2023 2 repositories listed
  In order to tackle both clip-level and frame-level tasks, this paper proposes Audio Teacher-Student Transformer (ATST), with a clip-level version (named ATST-Clip) and a frame-level version (named ATST-Frame),…
• Bytes Are All You Need: Transformers Operating Directly On File Bytes
  31 May 2023 2 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  Compared to Perceiver IO, our model requires absolutely no modality-specific processing at inference time, and uses an order of magnitude fewer parameters at equivalent accuracy on ImageNet.
• ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities
  18 May 2023 2 repositories listed Syntology ran 2 of 7 samples · 5 unverified
  In this work, we explore a scalable way for building a general representation model toward unlimited modalities.
• Audiovisual Masked Autoencoders
  9 Dec 2022 2 repositories listed
  Can we leverage the audiovisual information already present in video to improve self-supervised representation learning?
• Efficient Large-scale Audio Tagging via Transformer-to-CNN Knowledge Distillation
  9 Nov 2022 2 repositories listed Syntology ran 1 of 2 samples · 1 unverified
  We provide models of different complexity levels, scaling from low-complexity models up to a new state-of-the-art performance of .

Syntology lines on 20 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.