Audio Generation

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

3 leaderboard tables shown for this task, 3 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

11 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

4 subtasks in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 124 papers with code (270 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• WaveNet: A Generative Model for Raw Audio
  12 Sep 2016 62 repositories listed Syntology ran 41 of 103 samples · 62 unverified · 25 pointer-only (licence)
  This paper introduces WaveNet, a deep neural network for generating raw audio waveforms.
• Adversarial Audio Synthesis
  12 Feb 2018 22 repositories listed Syntology ran 2 of 4 samples · 2 unverified · 4 pointer-only (licence)
  Audio signals are sampled at high temporal resolutions, and learning to synthesize audio requires capturing structure across a range of timescales.
• AudioLM: a Language Modeling Approach to Audio Generation
  7 Sep 2022 6 repositories listed Syntology ran 4 of 15 samples · 11 unverified · 3 pointer-only (licence)
  We introduce AudioLM, a framework for high-quality audio generation with long-term consistency.
• It's Raw! Audio Generation with State-Space Models
  20 Feb 2022 6 repositories listed Syntology ran 6 of 6 samples · 0 unverified · 4 pointer-only (licence)
  SaShiMi yields state-of-the-art performance for unconditional waveform generation in the autoregressive setting.
• GANSynth: Adversarial Neural Audio Synthesis
  23 Feb 2019 6 repositories listed
  Efficient audio synthesis is an inherently difficult machine learning task, as human perception is sensitive to both global structure and fine-scale waveform coherence.
• BigVGAN: A Universal Neural Vocoder with Large-Scale Training
  9 Jun 2022 5 repositories listed Syntology ran 8 of 17 samples · 9 unverified · 1 pointer-only (licence)
  Despite recent progress in generative adversarial network (GAN)-based vocoders, where the model generates raw waveform conditioned on acoustic features, it is challenging to synthesize high-fidelity audio for numerous…
• MelNet: A Generative Model for Audio in the Frequency Domain
  4 Jun 2019 5 repositories listed Syntology ran 4 of 4 samples · 0 unverified · 4 pointer-only (licence)
  Capturing high-level structure in audio waveforms is challenging because a single second of audio spans tens of thousands of timesteps.
• High-Fidelity Audio Compression with Improved RVQGAN
  11 Jun 2023 4 repositories listed Syntology ran 27 of 38 samples · 11 unverified
  Language models have been successfully used to model natural signals, such as images, speech, and music.
• AudioLDM: Text-to-Audio Generation with Latent Diffusion Models
  29 Jan 2023 4 repositories listed Syntology ran 8 of 21 samples · 13 unverified · 10 pointer-only (licence)
  By learning the latent representations of audio signals and their compositions without modeling the cross-modal relationship, AudioLDM is advantageous in both generation quality and computational efficiency.
• Audio Super Resolution using Neural Networks
  2 Aug 2017 4 repositories listed Syntology ran 4 of 4 samples · 0 unverified · 3 pointer-only (licence)
  We introduce a new audio processing technique that increases the sampling rate of signals such as speech or music using deep convolutional neural networks.
• SampleRNN: An Unconditional End-to-End Neural Audio Generation Model
  22 Dec 2016 4 repositories listed Syntology ran 0 of 11 samples · 11 unverified
  In this paper we propose a novel model for unconditional audio generation based on generating one audio sample at a time.
• SoundStorm: Efficient Parallel Audio Generation
  16 May 2023 3 repositories listed Syntology ran 6 of 11 samples · 5 unverified · 3 pointer-only (licence)
  We present SoundStorm, a model for efficient, non-autoregressive audio generation.
• Enhancing Suno's Bark Text-to-Speech Model: Addressing Limitations Through Meta's Encodec and Pre-Trained Hubert
  18 Apr 2023 3 repositories listed
  Keywords: Bark, ai voice cloning, Suno, text-to-speech, artificial intelligence, audio generation, Meta's encodec, audio codebooks, semantic tokens, HuBert, transformer-based model, multilingual speech, wav2vec, linear…
• Differentiable Time-Frequency Scattering on GPU
  18 Apr 2022 3 repositories listed
  Joint time-frequency scattering (JTFS) is a convolutional operator in the time-frequency domain which extracts spectrotemporal modulations at various rates and scales.
• HiFi++: a Unified Framework for Bandwidth Extension and Speech Enhancement
  24 Mar 2022 3 repositories listed Syntology ran 9 of 12 samples · 3 unverified · 1 pointer-only (licence)
  Generative adversarial networks have recently demonstrated outstanding performance in neural vocoding outperforming best autoregressive and flow-based models.
• Taming Visually Guided Sound Generation
  17 Oct 2021 3 repositories listed Syntology ran 1 of 7 samples · 6 unverified
  In this work, we propose a single model capable of generating visually relevant, high-fidelity sounds prompted with a set of frames from open-domain videos in less time than it takes to play it on a single GPU.
• DDSP: Differentiable Digital Signal Processing
  14 Jan 2020 3 repositories listed Syntology ran 0 of 5 samples · 5 unverified · 5 pointer-only (licence)
  In this paper, we introduce the Differentiable Digital Signal Processing (DDSP) library, which enables direct integration of classic signal processing elements with deep learning methods.
• Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversion
  3 Jun 2019 3 repositories listed Syntology ran 1 of 5 samples · 4 unverified
  End-to-end models for raw audio generation are a challenge, specially if they have to work with non-parallel data, which is a desirable setup in many situations.
• Assisted Sound Sample Generation with Musical Conditioning in Adversarial Auto-Encoders
  12 Apr 2019 3 repositories listed
  Its training data subsets can directly be visualized in the 3D latent representation.
• Latent Swap Joint Diffusion for 2D Long-Form Latent Generation
  7 Feb 2025 2 repositories listed
  To address this issue, we propose Self-Loop Latent Swap, a frame-level bidirectional swap applied to the overlapping region of adjacent views.
• Audio Deepfake Detection with Self-Supervised XLS-R and SLS Classifier
  28 Oct 2024 2 repositories listed
  To enhance the sensitivity of deepfake audio features, we propose a deepfake audio detection model that incorporates an SLS (Sensitive Layer Selection) module.
• Movie Gen: A Cast of Media Foundation Models
  17 Oct 2024 2 repositories listed
  Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation.
• ESPnet-Codec: Comprehensive Training and Evaluation of Neural Codecs for Audio, Music, and Speech
  24 Sep 2024 2 repositories listed
  Neural codecs have become crucial to recent speech and audio generation research.
• AudioLCM: Text-to-Audio Generation with Latent Consistency Models
  1 Jun 2024 2 repositories listed
  To overcome the convergence issue inherent in LDMs with reduced sample iterations, we propose the Guided Latent Consistency Distillation with a multi-step Ordinary Differential Equation (ODE) solver.
• Fast Timing-Conditioned Latent Audio Diffusion
  7 Feb 2024 2 repositories listed Syntology ran 2 of 4 samples · 2 unverified
  Generating long-form 44.
• Invisible Watermarking for Audio Generation Diffusion Models
  22 Sep 2023 2 repositories listed
  Diffusion models have gained prominence in the image domain for their capabilities in data generation and transformation, achieving state-of-the-art performance in various tasks in both image and audio domains.
• AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining
  10 Aug 2023 2 repositories listed Syntology ran 12 of 27 samples · 15 unverified · 13 pointer-only (licence)
  Any audio can be translated into LOA based on AudioMAE, a self-supervised pre-trained representation learning model.
• MuseCoco: Generating Symbolic Music from Text
  31 May 2023 2 repositories listed
  In contrast, symbolic music offers ease of editing, making it more accessible for users to manipulate specific musical elements.
• Any-to-Any Generation via Composable Diffusion
  19 May 2023 2 repositories listed Syntology ran 2 of 13 samples · 11 unverified
  We present Composable Diffusion (CoDi), a novel generative model capable of generating any combination of output modalities, such as language, image, video, or audio, from any combination of input modalities.
• Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpus
  20 Dec 2021 2 repositories listed Syntology ran 0 of 2 samples · 2 unverified
  High-fidelity multi-singer singing voice synthesis is challenging for neural vocoder due to the singing voice data shortage, limited singer generalization, and large computational cost.

Syntology lines on 19 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.