LILAC: An Idempotent Neural Speech Codec

• URL: http://arxiv.org/abs/2608.05727v1
• Date: Thu, 06 Aug 2026 08:09:48 GMT
• Title: LILAC: An Idempotent Neural Speech Codec
• Abstract summary: LILAC is a fully convolutional 24 kHz speech at 9.375 Hz and 0.75 kbit/s that is idempotent by construction.<n>We present LILAC, a fully convolutional 24 kHz speech at 9.375 Hz and 0.75 kbit/s that is idempotent by construction.
• Score: 44.07258358805709
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural Audio Codecs are widely adopted in speech generation and editing. However, existing neural audio codecs are not idempotent: across the paper's twelve baseline systems, every configuration tested rewrites, on average, at least 15% of its tokens in a single decode-re-encode pass. This poses a problem for utilizing Neural Audio Codecs as token interfaces in pipelines where re-encoding decoded outputs can occur. We present LILAC, a fully convolutional 24 kHz speech codec at 9.375 Hz and 0.75 kbit/s that is codec idempotent by construction; re-encoding the decoded audio of any valid token stream returns the identical stream. LILAC achieves idempotency while maintaining competitive quality, reaching UTMOS 4.14 and 4.24 on LibriSpeech and LibriTTS-R test sets, comparable to SOTA sub-1 kbit/s Neural Audio Codecs.

Related papers

• CleanCodec: Efficient and Robust Speech Tokenization via Perceptually Guided Encoding [15.228841447073668]
  CleanCodec learns to encode only perceptually important features and discard imperceptible information.<n>At just 12.5 tokens per second, CleanCodec achieves state-of-the-art tokenization efficiency.
  arXiv  Detail & Related papers  (2026-06-03T03:56:14Z)
• EntangleCodec: A Unified Discrete Audio Tokenizer via Semantic-Acoustic Entanglement [54.95940885045993]
  EntangleCodec is a unified discrete audio tokenizer that learns caption-aligned semantic-acoustic representations before quantization.<n>It captures linguistic content, speaker identity, emotion, prosody, and acoustic scenes within a compact token stream.<n>It achieves reconstruction quality competitive with specialized codecs, and supports both TTS and TTA generation in a unified framework.
  arXiv  Detail & Related papers  (2026-06-01T18:05:18Z)
• How to Label Resynthesized Audio: The Dual Role of Neural Audio Codecs in Audio Deepfake Detection [60.88800374832363]
  Recent spoof detection studies use resynthesized waveforms from vocoders and neural audio codecs to simulate an attacker.<n>We examine how different labeling choices affect detection performance and provide insights into labeling strategies.
  arXiv  Detail & Related papers  (2026-02-18T10:29:07Z)
• FocalCodec-Stream: Streaming Low-Bitrate Speech Coding via Causal Distillation [27.32235541083431]
  FocalCodec-Stream is a hybrid that compresses speech into a single binary codebook at 0.55 - 0.80 kbps with a theoretical latency of 80 ms.<n> Experiments show that FocalCodec-Stream outperforms existing streamable codecs at comparables.
  arXiv  Detail & Related papers  (2025-09-19T17:57:13Z)
• Finite Scalar Quantization Enables Redundant and Transmission-Robust Neural Audio Compression at Low Bit-rates [1.445167946386569]
  We show that Finite Scalar Quantization (FSQ) encodes baked-in redundancy which produces an encoding which is robust when transmitted through noisy channels.<n>We demonstrate that FSQ has vastly superior bit-level perturbation by comparing the performance of RVQ and FSQ codecs when simulating the transmission of code sequences through a noisy channel.
  arXiv  Detail & Related papers  (2025-09-11T15:39:59Z)
• SecoustiCodec: Cross-Modal Aligned Streaming Single-Codecbook Speech Codec [83.61175662066364]
  Speech codecs serve as a crucial bridge in unifying speech and text language models.<n>Existing methods face several challenges in semantic encoding.<n>We propose SecoustiCodec, a cross-modal aligned low-bitrate streaming speech codecs.
  arXiv  Detail & Related papers  (2025-08-04T19:22:14Z)
• FlowDec: A flow-based full-band general audio codec with high perceptual quality [90.05968801459524]
  FlowDec is a neural full-band audio codecs for general audio sampled at 48 kHz.<n>We generalize from speech to general audio and move from 24 kbit/s to as low as 4 kbit/s.
  arXiv  Detail & Related papers  (2025-03-03T12:49:09Z)
• A Closer Look at Neural Codec Resynthesis: Bridging the Gap between Codec and Waveform Generation [65.05719674893999]
  We study two different strategies based on token prediction and regression, and introduce a new method based on Schr"odinger Bridge. We examine how different design choices affect machine and human perception.
  arXiv  Detail & Related papers  (2024-10-29T18:29:39Z)
• SNAC: Multi-Scale Neural Audio Codec [1.0753191494611891]
  Multi-Scale Neural Audio Codec is a simple extension of RVQ where the quantizers can operate at different temporal resolutions. This paper proposes Multi-Scale Neural Audio Codec, a simple extension of RVQ where the quantizers can operate at different temporal resolutions.
  arXiv  Detail & Related papers  (2024-10-18T12:24:05Z)
• SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound [40.810505707522324]
  SemantiCodec is designed to compress audio into fewer than a hundred tokens per second across diverse audio types.<n>We show that SemantiCodec significantly outperforms the state-of-the-art Descript on reconstruction quality.<n>Our results also suggest that SemantiCodec contains significantly richer semantic information than all evaluated state-of-the-art audio codecs.
  arXiv  Detail & Related papers  (2024-04-30T22:51:36Z)
• SoundStream: An End-to-End Neural Audio Codec [78.94923131038682]
  We present SoundStream, a novel neural audio system that can efficiently compress speech, music and general audio. SoundStream relies on a fully convolutional encoder/decoder network and a residual vector quantizer, which are trained jointly end-to-end. We are able to perform joint compression and enhancement either at the encoder or at the decoder side with no additional latency.
  arXiv  Detail & Related papers  (2021-07-07T15:45:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.