Diff-TTS: A Denoising Diffusion Model for Text-to-Speech

• URL: http://arxiv.org/abs/2104.01409v1
• Date: Sat, 3 Apr 2021 13:53:19 GMT
• Title: Diff-TTS: A Denoising Diffusion Model for Text-to-Speech
• Authors: Myeonghun Jeong, Hyeongju Kim, Sung Jun Cheon, Byoung Jin Choi, and Nam Soo Kim
• Abstract summary: We propose a novel non-autoregressive TTS model, namely Diff-TTS, which achieves highly natural and efficient speech synthesis. Given the text, Diff-TTS exploits a denoising diffusion framework to transform the noise signal into a mel-spectrogram via diffusion time steps. We verify that Diff-TTS generates 28 times faster than the real-time with a single NVIDIA 2080Ti GPU.
• Score: 14.231478930274058
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although neural text-to-speech (TTS) models have attracted a lot of attention and succeeded in generating human-like speech, there is still room for improvements to its naturalness and architectural efficiency. In this work, we propose a novel non-autoregressive TTS model, namely Diff-TTS, which achieves highly natural and efficient speech synthesis. Given the text, Diff-TTS exploits a denoising diffusion framework to transform the noise signal into a mel-spectrogram via diffusion time steps. In order to learn the mel-spectrogram distribution conditioned on the text, we present a likelihood-based optimization method for TTS. Furthermore, to boost up the inference speed, we leverage the accelerated sampling method that allows Diff-TTS to generate raw waveforms much faster without significantly degrading perceptual quality. Through experiments, we verified that Diff-TTS generates 28 times faster than the real-time with a single NVIDIA 2080Ti GPU.

Related papers

• SimpleSpeech 2: Towards Simple and Efficient Text-to-Speech with Flow-based Scalar Latent Transformer Diffusion Models [64.40250409933752]
  We build upon our previous publication by implementing a simple and efficient non-autoregressive (NAR) TTS framework, termed SimpleSpeech 2. SimpleSpeech 2 effectively combines the strengths of both autoregressive (AR) and non-autoregressive (NAR) methods. We show a significant improvement in generation performance and generation speed compared to our previous work and other state-of-the-art (SOTA) large-scale TTS models.
  arXiv  Detail & Related papers  (2024-08-25T17:07:39Z)
• Schrodinger Bridges Beat Diffusion Models on Text-to-Speech Synthesis [35.16243386407448]
  Bridge-TTS is a novel TTS system that substitutes the noisy Gaussian prior in established diffusion-based TTS methods with a clean and deterministic one. Specifically, we leverage the latent representation obtained from text input as our prior, and build a fully tractable Schrodinger bridge between it and the ground-truth mel-spectrogram.
  arXiv  Detail & Related papers  (2023-12-06T13:31:55Z)
• High-Fidelity Speech Synthesis with Minimal Supervision: All Using Diffusion Models [56.00939852727501]
  Minimally-supervised speech synthesis decouples TTS by combining two types of discrete speech representations. Non-autoregressive framework enhances controllability, and duration diffusion model enables diversified prosodic expression.
  arXiv  Detail & Related papers  (2023-09-27T09:27:03Z)
• Boosting Fast and High-Quality Speech Synthesis with Linear Diffusion [85.54515118077825]
  This paper proposes a linear diffusion model (LinDiff) based on an ordinary differential equation to simultaneously reach fast inference and high sample quality. To reduce computational complexity, LinDiff employs a patch-based processing approach that partitions the input signal into small patches. Our model can synthesize speech of a quality comparable to that of autoregressive models with faster synthesis speed.
  arXiv  Detail & Related papers  (2023-06-09T07:02:43Z)
• ResGrad: Residual Denoising Diffusion Probabilistic Models for Text to Speech [37.29193613404699]
  DDPMs are emerging in text-to-speech (TTS) synthesis because of their strong capability of generating high-fidelity samples. Previous works have explored speeding up inference speed by minimizing the number of inference steps but at the cost of sample quality. We propose ResGrad, a lightweight diffusion model which learns to refine the output spectrogram of an existing TTS model.
  arXiv  Detail & Related papers  (2022-12-30T02:31:35Z)
• ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech [63.780196620966905]
  We propose ProDiff, on progressive fast diffusion model for high-quality text-to-speech. ProDiff parameterizes the denoising model by directly predicting clean data to avoid distinct quality degradation in accelerating sampling. Our evaluation demonstrates that ProDiff needs only 2 iterations to synthesize high-fidelity mel-spectrograms. ProDiff enables a sampling speed of 24x faster than real-time on a single NVIDIA 2080Ti GPU.
  arXiv  Detail & Related papers  (2022-07-13T17:45:43Z)
• FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis [90.3069686272524]
  This paper proposes FastDiff, a fast conditional diffusion model for high-quality speech synthesis. FastDiff employs a stack of time-aware location-variable convolutions of diverse receptive field patterns to efficiently model long-term time dependencies. Based on FastDiff, we design an end-to-end text-to-speech synthesizer, FastDiff-TTS, which generates high-fidelity speech waveforms.
  arXiv  Detail & Related papers  (2022-04-21T07:49:09Z)
• DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs [39.388599580262614]
  We introduce DiffGAN-TTS, a novel DDPM-based text-to-speech (TTS) model achieving high-fidelity speech synthesis. Our experiments show that DiffGAN-TTS can achieve high synthesis performance with only 1 denoising step.
  arXiv  Detail & Related papers  (2022-01-28T07:41:10Z)
• A Study on Speech Enhancement Based on Diffusion Probabilistic Model [63.38586161802788]
  We propose a diffusion probabilistic model-based speech enhancement model (DiffuSE) model that aims to recover clean speech signals from noisy signals. The experimental results show that DiffuSE yields performance that is comparable to related audio generative models on the standardized Voice Bank corpus task.
  arXiv  Detail & Related papers  (2021-07-25T19:23:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.