HiFi-VC: High Quality ASR-Based Voice Conversion

• URL: http://arxiv.org/abs/2203.16937v1
• Date: Thu, 31 Mar 2022 10:45:32 GMT
• Title: HiFi-VC: High Quality ASR-Based Voice Conversion
• Authors: A. Kashkin, I. Karpukhin, S. Shishkin
• Abstract summary: We propose a new any-to-any voice conversion pipeline. Our approach uses automated speech recognition features, pitch tracking, and a state-of-the-art waveform prediction model.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The goal of voice conversion (VC) is to convert input voice to match the target speaker's voice while keeping text and prosody intact. VC is usually used in entertainment and speaking-aid systems, as well as applied for speech data generation and augmentation. The development of any-to-any VC systems, which are capable of generating voices unseen during model training, is of particular interest to both researchers and the industry. Despite recent progress, any-to-any conversion quality is still inferior to natural speech. In this work, we propose a new any-to-any voice conversion pipeline. Our approach uses automated speech recognition (ASR) features, pitch tracking, and a state-of-the-art waveform prediction model. According to multiple subjective and objective evaluations, our method outperforms modern baselines in terms of voice quality, similarity and consistency.

Related papers

• SEF-VC: Speaker Embedding Free Zero-Shot Voice Conversion with Cross Attention [24.842378497026154]
  SEF-VC is a speaker embedding free voice conversion model. It learns and incorporates speaker timbre from reference speech via a powerful position-agnostic cross-attention mechanism. It reconstructs waveform from HuBERT semantic tokens in a non-autoregressive manner.
  arXiv  Detail & Related papers  (2023-12-14T06:26:55Z)
• Non-Parallel Voice Conversion for ASR Augmentation [23.95732033698818]
  Voice conversion can be used as a data augmentation technique to improve ASR performance. Despite including many speakers, speaker diversity may remain a limitation to ASR quality.
  arXiv  Detail & Related papers  (2022-09-15T00:40:35Z)
• Robust Disentangled Variational Speech Representation Learning for Zero-shot Voice Conversion [34.139871476234205]
  We investigate zero-shot voice conversion from a novel perspective of self-supervised disentangled speech representation learning. A zero-shot voice conversion is performed by feeding an arbitrary speaker embedding and content embeddings to a sequential variational autoencoder (VAE) decoder. On TIMIT and VCTK datasets, we achieve state-of-the-art performance on both objective evaluation, i.e., speaker verification (SV) on speaker embedding and content embedding, and subjective evaluation, i.e. voice naturalness and similarity, and remains to be robust even with noisy source/target utterances.
  arXiv  Detail & Related papers  (2022-03-30T23:03:19Z)
• StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion [19.74933410443264]
  We present an unsupervised many-to-many voice conversion (VC) method using a generative adversarial network (GAN) called StarGAN v2. Our model is trained only with 20 English speakers. It generalizes to a variety of voice conversion tasks, such as any-to-many, cross-lingual, and singing conversion.
  arXiv  Detail & Related papers  (2021-07-21T23:44:17Z)
• On Prosody Modeling for ASR+TTS based Voice Conversion [82.65378387724641]
  In voice conversion, an approach showing promising results in the latest voice conversion challenge (VCC) 2020 is to first use an automatic speech recognition (ASR) model to transcribe the source speech into the underlying linguistic contents. Such a paradigm, referred to as ASR+TTS, overlooks the modeling of prosody, which plays an important role in speech naturalness and conversion similarity. We propose to directly predict prosody from the linguistic representation in a target-speaker-dependent manner, referred to as target text prediction (TTP)
  arXiv  Detail & Related papers  (2021-07-20T13:30:23Z)
• VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversion [54.29557210925752]
  One-shot voice conversion can be effectively achieved by speech representation disentanglement. We employ vector quantization (VQ) for content encoding and introduce mutual information (MI) as the correlation metric during training. Experimental results reflect the superiority of the proposed method in learning effective disentangled speech representations.
  arXiv  Detail & Related papers  (2021-06-18T13:50:38Z)
• Voicy: Zero-Shot Non-Parallel Voice Conversion in Noisy Reverberant Environments [76.98764900754111]
  Voice Conversion (VC) is a technique that aims to transform the non-linguistic information of a source utterance to change the perceived identity of the speaker. We propose Voicy, a new VC framework particularly tailored for noisy speech. Our method, which is inspired by the de-noising auto-encoders framework, is comprised of four encoders (speaker, content, phonetic and acoustic-ASR) and one decoder.
  arXiv  Detail & Related papers  (2021-06-16T15:47:06Z)
• An Adaptive Learning based Generative Adversarial Network for One-To-One Voice Conversion [9.703390665821463]
  We propose an adaptive learning-based GAN model called ALGAN-VC for an efficient one-to-one VC of speakers. The model is tested on Voice Conversion Challenge (VCC) 2016, 2018, and 2020 datasets as well as on our self-prepared speech dataset. A subjective and objective evaluation of the generated speech samples indicated that the proposed model elegantly performed the voice conversion task.
  arXiv  Detail & Related papers  (2021-04-25T13:44:32Z)
• The Sequence-to-Sequence Baseline for the Voice Conversion Challenge 2020: Cascading ASR and TTS [66.06385966689965]
  This paper presents the sequence-to-sequence (seq2seq) baseline system for the voice conversion challenge (VCC) 2020. We consider a naive approach for voice conversion (VC), which is to first transcribe the input speech with an automatic speech recognition (ASR) model. We revisit this method under a sequence-to-sequence (seq2seq) framework by utilizing ESPnet, an open-source end-to-end speech processing toolkit.
  arXiv  Detail & Related papers  (2020-10-06T02:27:38Z)
• VQVC+: One-Shot Voice Conversion by Vector Quantization and U-Net architecture [71.45920122349628]
  Auto-encoder-based VC methods disentangle the speaker and the content in input speech without given the speaker's identity. We use the U-Net architecture within an auto-encoder-based VC system to improve audio quality.
  arXiv  Detail & Related papers  (2020-06-07T14:01:16Z)
• VoiceCoach: Interactive Evidence-based Training for Voice Modulation Skills in Public Speaking [55.366941476863644]
  The modulation of voice properties, such as pitch, volume, and speed, is crucial for delivering a successful public speech. We present VoiceCoach, an interactive evidence-based approach to facilitate the effective training of voice modulation skills.
  arXiv  Detail & Related papers  (2020-01-22T04:52:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.