Kanade: A Simple Disentangled Tokenizer for Spoken Language Modeling

• URL: http://arxiv.org/abs/2602.00594v1
• Date: Sat, 31 Jan 2026 08:20:39 GMT
• Title: Kanade: A Simple Disentangled Tokenizer for Spoken Language Modeling
• Authors: Zhijie Huang, Stephen McIntosh, Daisuke Saito, Nobuaki Minematsu,
• Abstract summary: Tokenization is important for speech modeling, which must handle continuous signals that mix linguistic and non-linguistic information.<n>We present Kanade, a single-layer disentangled speech tokenizer.<n>Experiments show that Kanade achieves state-of-the-art speaker disentanglement and lexical availability, while maintaining excellent reconstruction quality.
• Score: 17.56787468222101
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: A good language model starts with a good tokenizer. Tokenization is especially important for speech modeling, which must handle continuous signals that mix linguistic and non-linguistic information. A speech tokenizer should extract phonetics and prosody, suppress linguistically irrelevant information like speaker identity, and enable high-quality synthesis. We present Kanade, a single-layer disentangled speech tokenizer that realizes this ideal. Kanade separates out acoustic constants to create a single stream of tokens that captures rich phonetics and prosody. It does so without the need for auxiliary methods that existing disentangled codecs often rely on. Experiments show that Kanade achieves state-of-the-art speaker disentanglement and lexical availability, while maintaining excellent reconstruction quality.

Related papers

• DSA-Tokenizer: Disentangled Semantic-Acoustic Tokenization via Flow Matching-based Hierarchical Fusion [28.204167153140506]
  Speech tokenizers serve as the cornerstone of discrete Speech Large Language Models.<n>We propose DSA-Tokenizer, which explicitly disentangles speech into discrete semantic and acoustic tokens.
  arXiv  Detail & Related papers  (2026-01-14T07:22:24Z)
• ProsodyLM: Uncovering the Emerging Prosody Processing Capabilities in Speech Language Models [70.56468982313834]
  We propose ProsodyLM, which introduces a simple tokenization scheme amenable to learning prosody.<n>We find that ProsodyLM can learn surprisingly diverse emerging prosody processing capabilities through pre-training alone.
  arXiv  Detail & Related papers  (2025-07-27T00:59:01Z)
• Factorized RVQ-GAN For Disentangled Speech Tokenization [34.73812744103814]
  We propose a unified neural speech encoder that factorizes its bottleneck into three linguistic levels-acoustic, phonetic, and lexical-within a single model.<n> Experiments on English and multilingual data show that HAC's factorized bottleneck yields disentangled token sets.<n> Quantitative evaluations confirm that HAC preserve naturalness and provide interpretable linguistic information, outperforming single-level baselines in both disentanglement and reconstruction quality.
  arXiv  Detail & Related papers  (2025-06-18T13:36:34Z)
• A Variational Framework for Improving Naturalness in Generative Spoken Language Models [52.673912922590866]
  We propose an end-to-end variational approach that automatically learns to encode continuous speech attributes to enhance semantic tokens.<n>Our approach eliminates the need for manual extraction and selection of paralinguistic features.<n>It produces preferred speech continuations according to human raters.
  arXiv  Detail & Related papers  (2025-06-17T17:58:17Z)
• Universal Speech Token Learning via Low-Bitrate Neural Codec and Pretrained Representations [32.83743219965261]
  This paper unifies two types of tokens and proposes the UniCodec, a universal speech token learning that encapsulates all semantics of speech.<n>A low-bitrate neural is leveraged to learn such disentangled discrete representations at global and local scales, with knowledge distilled from self-supervised learned features.
  arXiv  Detail & Related papers  (2025-03-15T12:50:43Z)
• CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models [74.80386066714229]
  We present an improved streaming speech synthesis model, CosyVoice 2.<n>Specifically, we introduce finite-scalar quantization to improve codebook utilization of speech tokens.<n>We develop a chunk-aware causal flow matching model to support various synthesis scenarios.
  arXiv  Detail & Related papers  (2024-12-13T12:59:39Z)
• Sylber: Syllabic Embedding Representation of Speech from Raw Audio [25.703703711031178]
  We propose a new model, Sylber, that produces speech representations with clean and robust syllabic structure.<n>Specifically, we propose a self-supervised learning framework that bootstraps syllabic embeddings by distilling from its own initial unsupervised syllabic segmentation.<n>This results in a highly structured representation of speech features, offering three key benefits: 1) a fast, linear-time syllable segmentation algorithm, 2) efficient syllabic tokenization with an average of 4.27 tokens per second, and 3) novel phonological units suited for efficient spoken language modeling.
  arXiv  Detail & Related papers  (2024-10-09T17:59:04Z)
• dMel: Speech Tokenization made Simple [16.679015298503593]
  We introduce a novel speech representation (dmel) that discretizes mel-filterbank channels into intensity bins.<n>Our approach demonstrates superior performance in preserving audio content, robustness to out-of-domain data, and offers a training-free, natural, and streamable representation.
  arXiv  Detail & Related papers  (2024-07-22T17:51:53Z)
• CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens [49.569695524535454]
  We propose to represent speech with supervised semantic tokens, which are derived from a multilingual speech recognition model by inserting vector quantization into the encoder. Based on the tokens, we further propose a scalable zero-shot TTS synthesizer, CosyVoice, which consists of an LLM for text-to-token generation and a conditional flow matching model for token-to-speech synthesis.
  arXiv  Detail & Related papers  (2024-07-07T15:16:19Z)
• SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models [58.996653700982556]
  Existing speech tokens are not specifically designed for speech language modeling. We propose SpeechTokenizer, a unified speech tokenizer for speech large language models. Experiments show that SpeechTokenizer performs comparably to EnCodec in speech reconstruction and demonstrates strong performance on the SLMTokBench benchmark.
  arXiv  Detail & Related papers  (2023-08-31T12:53:09Z)
• Bridging the Modality Gap for Speech-to-Text Translation [57.47099674461832]
  End-to-end speech translation aims to translate speech in one language into text in another language via an end-to-end way. Most existing methods employ an encoder-decoder structure with a single encoder to learn acoustic representation and semantic information simultaneously. We propose a Speech-to-Text Adaptation for Speech Translation model which aims to improve the end-to-end model performance by bridging the modality gap between speech and text.
  arXiv  Detail & Related papers  (2020-10-28T12:33:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.