Browse State-of-the-Art › Speech Representation Learning
Speech Representation Learning
48 papers with code · 0 benchmarks · 0 datasets archive 2025-07-28
Benchmarks archive 2025-07-28
No benchmark for this task in the archive.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
No dataset record in the archive lists this task.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 131 · with a code link: 48 · where Syntology ran a sample: 13 (7 with a run with no instrument failure, 6 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (13 of 131 tagged: 7 with a run with no instrument failure, 6 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 131, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
14 Jun 2021 11 repositories listed Syntology community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified (of 9 harvested samples)Self-supervised approaches for speech representation learning are challenged by three unique problems: (1) there are multiple sound units in each input utterance, (2) there is no lexicon of input sound units during the…
-
Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders25 Oct 2019 7 repositories listed Syntology community repositories only · 8 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 11 harvested samples) · 1 pointer-only (licence)We present Mockingjay as a new speech representation learning approach, where bidirectional Transformer encoders are pre-trained on a large amount of unlabeled speech.
-
12 Oct 2021 5 repositories listedWe integrate the proposed methods into the HuBERT framework.
-
19 Jan 2021 5 repositories listed Syntology official (archive's flag): 1 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)In this paper, we propose a unified pre-training approach called UniSpeech to learn speech representations with both unlabeled and labeled data, in which supervised phonetic CTC learning and phonetically-aware…
-
25 Jan 2019 5 repositories listedWe consider the task of unsupervised extraction of meaningful latent representations of speech by applying autoencoding neural networks to speech waveforms.
-
7 Aug 2021 4 repositories listedIn particular, when compared to published models such as conformer-based wav2vec 2.
-
5 Apr 2019 4 repositories listed Syntology 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 3 pointer-only (licence)This paper proposes a novel unsupervised autoregressive neural model for learning generic speech representations.
-
7 Nov 2022 2 repositories listedIn this paper, we extend the pretraining method for cross-lingual multi-speaker speech synthesis tasks, including cross-lingual multi-speaker voice cloning and cross-lingual multi-speaker speech editing.
-
18 Mar 2022 2 repositories listed Syntology 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified (of 7 harvested samples)Recently, speech representation learning has improved many speech-related tasks such as speech recognition, speech classification, and speech-to-text translation.
-
5 Jan 2022 2 repositories listedThe lip-reading WER is further reduced to 26.
-
17 Nov 2021 2 repositories listedOn the CoVoST-2 speech translation benchmark, we improve the previous state of the art by an average of 7.
-
30 Apr 2018 2 repositories listedWe apply these results to pairs of words discovered using an unsupervised algorithm and show an improvement on state-of-the-art in unsupervised representation learning using siamese networks.
-
23 Jan 2025 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)Specifically, we suggest a unimodal multi-task learning, which distills cross-modal knowledge and aligns the corrupted modalities, by predicting clean audio targets with corrupted videos, and clean video targets with…
-
26 Nov 2024 1 repository listedSelf-supervised learning (SSL) has achieved great success in speech-related tasks.
-
17 Oct 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified (of 10 harvested samples) · 2 pointer-only (licence)In this paper, we present EH-MAM (Easy-to-Hard adaptive Masked Acoustic Modeling), a novel self-supervised learning approach for speech representation learning.
-
16 Sep 2024 1 repository listedHowever, we observe that the information aggregated in the CLS token correlates more with speaker identity than with linguistic content.
-
10 Jun 2024 1 repository listedWe present mHuBERT-147, the first general-purpose massively multilingual HuBERT speech representation model trained on 90K hours of clean, open-license data.
-
13 Mar 2024 1 repository listedLastly, we show that the proposed recipe can be applied to other distillation methodologies, such as the recent DPWavLM.
-
21 Feb 2024 1 repository listedWe then show that the quality of the pre-trained model depends mainly on the amount of speech data seen during training, i. e., on the product of batch size and number of iterations.
-
18 Oct 2023 1 repository listedUsing a large multilingual audio corpus and self-supervised learning, CLARA develops speech representations enriched with emotions, advancing emotion-aware multilingual speech processing.
-
17 Oct 2023 1 repository listedIn this paper, we present a methodology for linguistic feature extraction, focusing particularly on automatically syllabifying words in multiple languages, with a design to be compatible with a forced-alignment tool…
-
25 Sep 2023 1 repository listedRecent years have witnessed significant advancements in self-supervised learning (SSL) methods for speech-processing tasks.
-
31 Aug 2023 1 repository listedThis paper proposes a novel semi-supervised TTS framework, QS-TTS, to improve TTS quality with lower supervised data requirements via Vector-Quantized Self-Supervised Speech Representation Learning (VQ-S3RL) utilizing…
-
17 May 2023 1 repository listedIn this paper, we introduce self-distillation and online clustering for self-supervised speech representation learning (DinoSR) which combines masked language modeling, self-distillation, and online clustering.
-
5 May 2023 1 repository listedThe latent space is structured to dissociate the latent dynamical factors that are shared between the modalities from those that are specific to each modality.
-
9 Mar 2023 1 repository listedThis paper presents FaceXHuBERT, a text-less speech-driven 3D facial animation generation method that allows to capture personalized and subtle cues in speech (e. g. identity, emotion and hesitation).
-
27 Feb 2023 1 repository listedThe SA represents our proposal for an efficient streaming SSRL implementation, while the LLSA solves the latency build-up problem of other streaming attention architectures, such as the masked acausal attention (MAA)…
-
27 Feb 2023 1 repository listedSelf-supervised speech representation learning (SSL) has shown to be effective in various downstream tasks, but SSL models are usually large and slow.
-
14 Nov 2022 1 repository listedIn this paper, we provide a new perspective on self-supervised speech models from how the training targets are obtained.
-
2 Nov 2022 1 repository listedIn this paper, we propose a new Self-Supervised Learning (SSL) algorithm called data2vec-aqc, for speech representation learning from unlabeled speech data.
-
2 Nov 2022 1 repository listed
-
27 Oct 2022 1 repository listed
-
26 Sep 2022 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 13 unverified (of 15 harvested samples)
-
25 May 2022 1 repository listed
-
8 Apr 2022 1 repository listed
-
1 Apr 2022 1 repository listed
-
31 Mar 2022 1 repository listed Syntology official (archive's flag): 1 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
30 Mar 2022 1 repository listed
-
29 Mar 2022 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 8 harvested samples) · 1 pointer-only (licence)
-
5 Jan 2022 1 repository listed Syntology official: no sample here; runs from other or unrecorded repositories · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
5 Oct 2021 1 repository listed
-
1 Jul 2021 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
1 Jun 2021 1 repository listed
-
27 Apr 2021 1 repository listed
-
15 Apr 2021 1 repository listed
-
16 Mar 2021 1 repository listed
-
11 Dec 2020 1 repository listed
-
27 Oct 2020 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples)
-
HYFuse: Aligning Heterogeneous Speech Pre-Trained Representations in Hyperbolic Space for Speech Emotion Recognition3 Jun 2025 0 repositories listed
-
DuRep: Dual-Mode Speech Representation Learning via ASR-Aware Distillation26 May 2025 0 repositories listed
-
Universal Semantic Disentangled Privacy-preserving Speech Representation Learning19 May 2025 0 repositories listed
-
UniWav: Towards Unified Pre-training for Speech Representation Learning and Generation2 Mar 2025 0 repositories listed
-
JOOCI: a Framework for Learning Comprehensive Speech Representations14 Oct 2024 0 repositories listed
-
Are Music Foundation Models Better at Singing Voice Deepfake Detection? Far-Better Fuse them with Speech Foundation Models21 Sep 2024 0 repositories listed
-
Progressive Residual Extraction based Pre-training for Speech Representation Learning31 Aug 2024 0 repositories listed
-
Speech Representation Learning Revisited: The Necessity of Separate Learnable Parameters and Robust Data Augmentation20 Aug 2024 0 repositories listed
-
Towards the Next Frontier in Speech Representation Learning Using Disentanglement2 Jul 2024 0 repositories listed
-
Towards Robust Speech Representation Learning for Thousands of Languages30 Jun 2024 0 repositories listed
-
Emotion-Aware Speech Self-Supervised Representation Learning with Intensity Knowledge10 Jun 2024 0 repositories listed
-
XLAVS-R: Cross-Lingual Audio-Visual Speech Representation Learning for Noise-Robust Speech Perception21 Mar 2024 0 repositories listed
-
UNIT-DSR: Dysarthric Speech Reconstruction System Using Speech Unit Normalization26 Jan 2024 0 repositories listed
-
Revisiting Self-supervised Learning of Speech Representation from a Mutual Information Perspective16 Jan 2024 0 repositories listed
-
Efficiency-oriented approaches for self-supervised speech representation learning18 Dec 2023 0 repositories listed
-
Reimagining Speech: A Scoping Review of Deep Learning-Powered Voice Conversion14 Nov 2023 0 repositories listed
-
Learning Disentangled Speech Representations4 Nov 2023 0 repositories listed
-
Privacy-preserving Representation Learning for Speech Understanding26 Oct 2023 0 repositories listed
-
Spatial HuBERT: Self-supervised Spatial Speech Representation Learning for a Single Talker from Multi-channel Audio17 Oct 2023 0 repositories listed
-
Evaluating Self-Supervised Speech Representations for Indigenous American Languages5 Oct 2023 0 repositories listed
-
Speech representation learning: Learning bidirectional encoders with single-view, multi-view, and multi-task methods25 Jul 2023 0 repositories listed
-
MASR: Multi-label Aware Speech Representation20 Jul 2023 0 repositories listed
-
On-Device Constrained Self-Supervised Speech Representation Learning for Keyword Spotting via Knowledge Distillation6 Jul 2023 0 repositories listed
-
Flowchase: a Mobile Application for Pronunciation Training5 Jul 2023 0 repositories listed
-
Label Aware Speech Representation Learning For Language Identification7 Jun 2023 0 repositories listed
-
Simultaneous or Sequential Training? How Speech Representations Cooperate in a Multi-Task Self-Supervised Learning System5 Jun 2023 0 repositories listed
-
An empirical study on speech restoration guided by self supervised speech representation30 May 2023 0 repositories listed
-
INTapt: Information-Theoretic Adversarial Prompt Tuning for Enhanced Non-Native Speech Recognition25 May 2023 0 repositories listed
-
TranUSR: Phoneme-to-word Transcoder Based Unified Speech Representation Learning for Cross-lingual Speech Recognition23 May 2023 0 repositories listed
-
Learning Cross-lingual Visual Speech Representations14 Mar 2023 0 repositories listed
-
Self-supervised speech representation learning for keyword-spotting with light-weight transformers7 Mar 2023 0 repositories listed
-
Efficient Speech Representation Learning with Low-Bit Quantization14 Dec 2022 0 repositories listed
-
Improved Self-Supervised Multilingual Speech Representation Learning Combined with Auxiliary Language Information7 Dec 2022 0 repositories listed
-
Disentangled Feature Learning for Real-Time Neural Speech Coding22 Nov 2022 0 repositories listed
-
VATLM: Visual-Audio-Text Pre-Training with Unified Masked Prediction for Speech Representation Learning21 Nov 2022 0 repositories listed
-
Improving the Robustness of DistilHuBERT to Unseen Noisy Conditions via Data Augmentation, Curriculum Learning, and Multi-Task Enhancement12 Nov 2022 0 repositories listed
-
Application of Knowledge Distillation to Multi-task Speech Representation Learning29 Oct 2022 0 repositories listed
-
Disentangled Speech Representation Learning for One-Shot Cross-lingual Voice Conversion Using β-VAE25 Oct 2022 0 repositories listed
-
Improving Speech Representation Learning via Speech-level and Phoneme-level Masking Approach25 Oct 2022 0 repositories listed
-
SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning16 Oct 2022 0 repositories listed
-
Experiments on Turkish ASR with Self-Supervised Speech Representation Learning13 Oct 2022 0 repositories listed
-
On the Use of Semantically-Aligned Speech Representations for Spoken Language Understanding11 Oct 2022 0 repositories listed
-
Unsupervised TTS Acoustic Modeling for TTS with Conditional Disentangled Sequential VAE6 Jun 2022 0 repositories listed
-
Self-supervised models of audio effectively explain human cortical responses to speech27 May 2022 0 repositories listed
-
Self-Supervised Speech Representation Learning: A Review21 May 2022 0 repositories listed
-
SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representation17 May 2022 0 repositories listed
-
Automatic Pronunciation Assessment using Self-Supervised Speech Representation Learning8 Apr 2022 0 repositories listed
-
Disentangled Speech Representation Learning Based on Factorized Hierarchical Variational Autoencoder with Self-Supervised Objective5 Apr 2022 0 repositories listed
-
Robust Speaker Recognition with Transformers Using wav2vec 2.028 Mar 2022 0 repositories listed
-
Leveraging unsupervised and weakly-supervised data to improve direct speech-to-speech translation24 Mar 2022 0 repositories listed
-
XTREME-S: Evaluating Cross-lingual Speech Representations21 Mar 2022 0 repositories listed
-
Privacy-Preserving Speech Representation Learning using Vector Quantization15 Mar 2022 0 repositories listed
Syntology lines on 13 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections