Invertible Voice Conversion

• URL: http://arxiv.org/abs/2201.10687v1
• Date: Wed, 26 Jan 2022 00:25:27 GMT
• Title: Invertible Voice Conversion
• Authors: Zexin Cai, Ming Li
• Abstract summary: In this paper, we propose an invertible deep learning framework called INVVC for voice conversion. We develop an invertible framework that makes the source identity traceable. We apply the proposed framework to one-to-one voice conversion and many-to-one conversion using parallel training data.
• Score: 12.095003816544919
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose an invertible deep learning framework called INVVC for voice conversion. It is designed against the possible threats that inherently come along with voice conversion systems. Specifically, we develop an invertible framework that makes the source identity traceable. The framework is built on a series of invertible $1\times1$ convolutions and flows consisting of affine coupling layers. We apply the proposed framework to one-to-one voice conversion and many-to-one conversion using parallel training data. Experimental results show that this approach yields impressive performance on voice conversion and, moreover, the converted results can be reversed back to the source inputs utilizing the same parameters as in forwarding.

Related papers

• Zero-shot Voice Conversion with Diffusion Transformers [0.0]
  Zero-shot voice conversion aims to transform a source speech utterance to match the timbre of a reference speech from an unseen speaker. Traditional approaches struggle with timbre leakage, insufficient timbre representation, and mismatches between training and inference tasks. We propose Seed-VC, a novel framework that addresses these issues by introducing an external timbre shifter during training.
  arXiv  Detail & Related papers  (2024-11-15T04:43:44Z)
• Principled Paraphrase Generation with Parallel Corpora [52.78059089341062]
  We formalize the implicit similarity function induced by round-trip Machine Translation. We show that it is susceptible to non-paraphrase pairs sharing a single ambiguous translation. We design an alternative similarity metric that mitigates this issue.
  arXiv  Detail & Related papers  (2022-05-24T17:22:42Z)
• StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion [19.74933410443264]
  We present an unsupervised many-to-many voice conversion (VC) method using a generative adversarial network (GAN) called StarGAN v2. Our model is trained only with 20 English speakers. It generalizes to a variety of voice conversion tasks, such as any-to-many, cross-lingual, and singing conversion.
  arXiv  Detail & Related papers  (2021-07-21T23:44:17Z)
• Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modeling [61.351967629600594]
  This paper proposes an any-to-many location-relative, sequence-to-sequence (seq2seq), non-parallel voice conversion approach. In this approach, we combine a bottle-neck feature extractor (BNE) with a seq2seq synthesis module. Objective and subjective evaluations show that the proposed any-to-many approach has superior voice conversion performance in terms of both naturalness and speaker similarity.
  arXiv  Detail & Related papers  (2020-09-06T13:01:06Z)
• Spectrum and Prosody Conversion for Cross-lingual Voice Conversion with CycleGAN [81.79070894458322]
  Cross-lingual voice conversion aims to change source speaker's voice to sound like that of target speaker, when source and target speakers speak different languages. Previous studies on cross-lingual voice conversion mainly focus on spectral conversion with a linear transformation for F0 transfer. We propose the use of continuous wavelet transform (CWT) decomposition for F0 modeling. CWT provides a way to decompose a signal into different temporal scales that explain prosody in different time resolutions.
  arXiv  Detail & Related papers  (2020-08-11T07:29:55Z)
• Pretraining Techniques for Sequence-to-Sequence Voice Conversion [57.65753150356411]
  Sequence-to-sequence (seq2seq) voice conversion (VC) models are attractive owing to their ability to convert prosody. We propose to transfer knowledge from other speech processing tasks where large-scale corpora are easily available, typically text-to-speech (TTS) and automatic speech recognition (ASR) We argue that VC models with such pretrained ASR or TTS model parameters can generate effective hidden representations for high-fidelity, highly intelligible converted speech.
  arXiv  Detail & Related papers  (2020-08-07T11:02:07Z)
• Many-to-Many Voice Transformer Network [55.17770019619078]
  This paper proposes a voice conversion (VC) method based on a sequence-to-sequence (S2S) learning framework. It enables simultaneous conversion of the voice characteristics, pitch contour, and duration of input speech.
  arXiv  Detail & Related papers  (2020-05-18T04:02:08Z)
• End-to-End Whisper to Natural Speech Conversion using Modified Transformer Network [0.8399688944263843]
  We introduce whisper-to-natural-speech conversion using sequence-to-sequence approach. We investigate different features like Mel frequency cepstral coefficients and smoothed spectral features. The proposed networks are trained end-to-end using supervised approach for feature-to-feature transformation.
  arXiv  Detail & Related papers  (2020-04-20T14:47:46Z)
• Vocoder-free End-to-End Voice Conversion with Transformer Network [5.5792083698526405]
  Mel-frequency filter bank (MFB) based approaches have the advantage of learning speech compared to raw spectrum since MFB has less feature size. It is possible to only use the raw spectrum along with the phase to generate different style of voices with clear pronunciation. In this paper, we introduce a vocoder-free end-to-end voice conversion method using transformer network.
  arXiv  Detail & Related papers  (2020-02-05T06:19:24Z)
• Transforming Spectrum and Prosody for Emotional Voice Conversion with Non-Parallel Training Data [91.92456020841438]
  Many studies require parallel speech data between different emotional patterns, which is not practical in real life. We propose a CycleGAN network to find an optimal pseudo pair from non-parallel training data. We also study the use of continuous wavelet transform (CWT) to decompose F0 into ten temporal scales, that describes speech prosody at different time resolution.
  arXiv  Detail & Related papers  (2020-02-01T12:36:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.