Deep Learning Based Assessment of Synthetic Speech Naturalness

• URL: http://arxiv.org/abs/2104.11673v1
• Date: Fri, 23 Apr 2021 16:05:20 GMT
• Title: Deep Learning Based Assessment of Synthetic Speech Naturalness
• Authors: Gabriel Mittag, Sebastian M\"oller
• Abstract summary: We present a new objective prediction model for synthetic speech naturalness. It can be used to evaluate Text-To-Speech or Voice Conversion systems.
• Score: 14.463987018380468
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we present a new objective prediction model for synthetic speech naturalness. It can be used to evaluate Text-To-Speech or Voice Conversion systems and works language independently. The model is trained end-to-end and based on a CNN-LSTM network that previously showed to give good results for speech quality estimation. We trained and tested the model on 16 different datasets, such as from the Blizzard Challenge and the Voice Conversion Challenge. Further, we show that the reliability of deep learning-based naturalness prediction can be improved by transfer learning from speech quality prediction models that are trained on objective POLQA scores. The proposed model is made publicly available and can, for example, be used to evaluate different TTS system configurations.

Related papers

• A Suite for Acoustic Language Model Evaluation [20.802090523583196]
  We introduce SALMon, a novel evaluation suite encompassing background noise, emotion, speaker identity and room impulse response. We evaluate several speech language models on SALMon, thus highlighting the strengths and weaknesses of each evaluated method.
  arXiv  Detail & Related papers  (2024-09-11T17:34:52Z)
• Developing Acoustic Models for Automatic Speech Recognition in Swedish [6.5458610824731664]
  This paper is concerned with automatic continuous speech recognition using trainable systems. The aim of this work is to build acoustic models for spoken Swedish.
  arXiv  Detail & Related papers  (2024-04-25T12:03:14Z)
• NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality [123.97136358092585]
  We develop a TTS system called NaturalSpeech that achieves human-level quality on a benchmark dataset. Specifically, we leverage a variational autoencoder (VAE) for end-to-end text to waveform generation. Experiment evaluations on popular LJSpeech dataset show that our proposed NaturalSpeech achieves -0.01 CMOS to human recordings at the sentence level.
  arXiv  Detail & Related papers  (2022-05-09T16:57:35Z)
• LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech [67.88748572167309]
  We present LDNet, a unified framework for mean opinion score (MOS) prediction. We propose two inference methods that provide more stable results and efficient computation.
  arXiv  Detail & Related papers  (2021-10-18T08:52:31Z)
• A study on the efficacy of model pre-training in developing neural text-to-speech system [55.947807261757056]
  This study aims to understand better why and how model pre-training can positively contribute to TTS system performance. It is found that the TTS system could achieve comparable performance when the pre-training data is reduced to 1/8 of its original size.
  arXiv  Detail & Related papers  (2021-10-08T02:09:28Z)
• Factorized Neural Transducer for Efficient Language Model Adaptation [51.81097243306204]
  We propose a novel model, factorized neural Transducer, by factorizing the blank and vocabulary prediction. It is expected that this factorization can transfer the improvement of the standalone language model to the Transducer for speech recognition. We demonstrate that the proposed factorized neural Transducer yields 15% to 20% WER improvements when out-of-domain text data is used for language model adaptation.
  arXiv  Detail & Related papers  (2021-09-27T15:04:00Z)
• Direct speech-to-speech translation with discrete units [64.19830539866072]
  We present a direct speech-to-speech translation (S2ST) model that translates speech from one language to speech in another language without relying on intermediate text generation. We propose to predict the self-supervised discrete representations learned from an unlabeled speech corpus instead. When target text transcripts are available, we design a multitask learning framework with joint speech and text training that enables the model to generate dual mode output (speech and text) simultaneously in the same inference pass.
  arXiv  Detail & Related papers  (2021-07-12T17:40:43Z)
• Deep MOS Predictor for Synthetic Speech Using Cluster-Based Modeling [16.43844160498413]
  Several recent papers have proposed deep-learning-based assessment models. We propose three models using cluster-based modeling methods. We show that the GQT layer helps to predict human assessment better by automatically learning the task.
  arXiv  Detail & Related papers  (2020-08-09T11:14:19Z)
• Pretraining Techniques for Sequence-to-Sequence Voice Conversion [57.65753150356411]
  Sequence-to-sequence (seq2seq) voice conversion (VC) models are attractive owing to their ability to convert prosody. We propose to transfer knowledge from other speech processing tasks where large-scale corpora are easily available, typically text-to-speech (TTS) and automatic speech recognition (ASR) We argue that VC models with such pretrained ASR or TTS model parameters can generate effective hidden representations for high-fidelity, highly intelligible converted speech.
  arXiv  Detail & Related papers  (2020-08-07T11:02:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.