Browse State-of-the-Art › Speech Tokenization
Speech Tokenization
9 papers with code · 0 benchmarks · 0 datasets archive 2025-07-28
Speech tokenization is the task of representing speech signals as a sequence of discrete units. Such representations can be later used for various downstream tasks including automatic speech recognition, text-to-speech, etc. Such representation serves as the basis of Speech Language Models.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
No benchmark for this task in the archive.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
No dataset record in the archive lists this task.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 21 · with a code link: 9 · where Syntology ran a sample: 5 (3 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (5 of 21 tagged: 3 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 21 of 21, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from.
-
9 Apr 2025 2 repositories listed Syntology official (archive's flag): 13 ran · 14 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 0 honoured, 0 violated, 12 with no contract checked; 2 where Syntology's instrument failed) · 6 unverified (of 20 harvested samples) · 20 pointer-only (licence)To our knowledge, TASTE is the first end-to-end approach that utilizes a reconstruction objective to automatically learn a text-aligned speech tokenization and embedding suitable for spoken language modeling.
-
24 May 2025 1 repository listedRecent advances in Multimodal Large Language Models (MLLMs) have significantly enhanced the naturalness and flexibility of human computer interaction by enabling seamless understanding across text, vision, and audio…
-
21 Nov 2024 1 repository listedSpoken term detection (STD) is often hindered by reliance on frame-level features and the computationally intensive DTW-based template matching, limiting its practicality.
-
19 Oct 2024 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample)Recent advancements in speech-language models have yielded significant improvements in speech tokenization and synthesis.
-
9 Oct 2024 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 2 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples)To bridge this gap, we propose a new model, Sylber, that produces speech representations with clean and robust syllabic structure.
-
5 Oct 2024 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)For speech in particular, the high resolution of waveforms (16, 000 samples/second or more) presents a significant challenge as speech-based language models have had to use several times more tokens per word than…
-
16 Sep 2024 1 repository listedHowever, we observe that the information aggregated in the CLS token correlates more with speaker identity than with linguistic content.
-
22 Jul 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)Inspired by this success, researchers have investigated various compression-based speech tokenization methods to discretize continuous speech signals, enabling the application of language modeling techniques to discrete…
-
31 Aug 2023 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)To improve the performance of these discrete speech tokens, we present RepCodec, a novel speech representation codec for semantic speech tokenization.
-
LM-SPT: LM-Aligned Semantic Distillation for Speech Tokenization20 Jun 2025 0 repositories listedWith the rapid progress of speech language models (SLMs), discrete speech tokens have emerged as a core interface between speech and text, enabling unified modeling across modalities.
-
Factorized RVQ-GAN For Disentangled Speech Tokenization18 Jun 2025 0 repositories listedWe propose Hierarchical Audio Codec (HAC), a unified neural speech codec that factorizes its bottleneck into three linguistic levels-acoustic, phonetic, and lexical-within a single model.
-
Exploring the Effect of Segmentation and Vocabulary Size on Speech Tokenization for Speech Language Models23 May 2025 0 repositories listedThe purpose of speech tokenization is to transform a speech signal into a sequence of discrete representations, serving as the foundation for speech language models (SLMs).
-
Impact of Frame Rates on Speech Tokenizer: A Case Study on Mandarin and English20 May 2025 0 repositories listedThe speech tokenizer plays a crucial role in recent speech tasks, generally serving as a bridge between speech signals and language models.
-
UniWav: Towards Unified Pre-training for Speech Representation Learning and Generation2 Mar 2025 0 repositories listedPre-training and representation learning have been playing an increasingly important role in modern speech processing.
-
Recent Advances in Discrete Speech Tokens: A Review10 Feb 2025 0 repositories listedThe rapid advancement of speech generation technologies in the era of large language models (LLMs) has established discrete speech tokens as a foundational paradigm for speech representation.
-
DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Models31 Oct 2024 0 repositories listedSpoken language models (SLMs) have gained increasing attention with advancements in text-based, decoder-only language models.
-
5 Sep 2024 0 repositories listedSpeech tokenization serves as the foundation of speech language model (LM), enabling them to perform various tasks such as spoken language modeling, text-to-speech, speech-to-text, etc.
-
STAB: Speech Tokenizer Assessment Benchmark4 Sep 2024 0 repositories listedRepresenting speech as discrete tokens provides a framework for transforming speech into a format that closely resembles text, thus enabling the use of speech as an input to the widely successful large language models…
-
Discrete Multimodal Transformers with a Pretrained Large Language Model for Mixed-Supervision Speech Processing4 Jun 2024 0 repositories listedRecent work on discrete speech tokenization has paved the way for models that can seamlessly perform multiple tasks across modalities, e.g., speech recognition, text to speech, speech to speech translation.
-
Speech Language Models (SLMs) aim to learn language from raw audio, without textual resources.
-
BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data12 Feb 2024 0 repositories listedWe introduce a text-to-speech (TTS) model called BASE TTS, which stands for Big Adaptive Streamable TTS with Emergent abilities.
Syntology lines on 7 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections