Application of ASV for Voice Identification after VC and Duration Predictor Improvement in TTS Models

• URL: http://arxiv.org/abs/2406.19243v1
• Date: Thu, 27 Jun 2024 15:08:51 GMT
• Title: Application of ASV for Voice Identification after VC and Duration Predictor Improvement in TTS Models
• Authors: Borodin Kirill Nikolayevich, Kudryavtsev Vasiliy Dmitrievich, Mkrtchian Grach Maratovich, Gorodnichev Mikhail Genadievich, Korzh Dmitrii Sergeevich,
• Abstract summary: This paper presents a system for automatic speaker verification. The primary objective of our model is the extraction of embeddings from the target speaker's audio. This information is used in our multivoice TTS pipeline, which is currently under development.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: One of the most crucial components in the field of biometric security is the automatic speaker verification system, which is based on the speaker's voice. It is possible to utilise ASVs in isolation or in conjunction with other AI models. In the contemporary era, the quality and quantity of neural networks are increasing exponentially. Concurrently, there is a growing number of systems that aim to manipulate data through the use of voice conversion and text-to-speech models. The field of voice biometrics forgery is aided by a number of challenges, including SSTC, ASVSpoof, and SingFake. This paper presents a system for automatic speaker verification. The primary objective of our model is the extraction of embeddings from the target speaker's audio in order to obtain information about important characteristics of his voice, such as pitch, energy, and the duration of phonemes. This information is used in our multivoice TTS pipeline, which is currently under development. However, this model was employed within the SSTC challenge to verify users whose voice had undergone voice conversion, where it demonstrated an EER of 20.669.

Related papers

• SONAR: A Synthetic AI-Audio Detection Framework and Benchmark [59.09338266364506]
  SONAR is a synthetic AI-Audio Detection Framework and Benchmark. It aims to provide a comprehensive evaluation for distinguishing cutting-edge AI-synthesized auditory content. It is the first framework to uniformly benchmark AI-audio detection across both traditional and foundation model-based deepfake detection systems.
  arXiv  Detail & Related papers  (2024-10-06T01:03:42Z)
• Multi-modal Adversarial Training for Zero-Shot Voice Cloning [9.823246184635103]
  We propose a Transformer encoder-decoder architecture to conditionally discriminate between real and generated speech features. We introduce our novel adversarial training technique by applying it to a FastSpeech2 acoustic model and training on Libriheavy, a large multi-speaker dataset. Our model achieves improvements over the baseline in terms of speech quality and speaker similarity.
  arXiv  Detail & Related papers  (2024-08-28T16:30:41Z)
• Multi-speaker Text-to-speech Training with Speaker Anonymized Data [40.70515431989197]
  We investigate training multi-speaker text-to-speech (TTS) models using data that underwent speaker anonymization (SA) Two signal processing-based and three deep neural network-based SA methods were used to anonymize VCTK, a multi-speaker TTS dataset. We conducted extensive objective and subjective experiments to evaluate the anonymized training data, as well as the performance of the downstream TTS model trained using those data.
  arXiv  Detail & Related papers  (2024-05-20T03:55:44Z)
• NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers [90.83782600932567]
  We develop NaturalSpeech 2, a TTS system that leverages a neural audio predictor with residual vectorizers to get the quantized latent vectors. We scale NaturalSpeech 2 to large-scale datasets with 44K hours of speech and singing data and evaluate its voice quality on unseen speakers. NaturalSpeech 2 outperforms previous TTS systems by a large margin in terms of prosody/timbre similarity, synthesis, and voice quality in a zero-shot setting.
  arXiv  Detail & Related papers  (2023-04-18T16:31:59Z)
• Unsupervised TTS Acoustic Modeling for TTS with Conditional Disentangled Sequential VAE [36.50265124324876]
  We propose a novel unsupervised text-to-speech acoustic model training scheme, named UTTS, which does not require text-audio pairs. The framework offers a flexible choice of a speaker's duration model, timbre feature (identity) and content for TTS inference. Experiments demonstrate that UTTS can synthesize speech of high naturalness and intelligibility measured by human and objective evaluations.
  arXiv  Detail & Related papers  (2022-06-06T11:51:22Z)
• HiFi-VC: High Quality ASR-Based Voice Conversion [0.0]
  We propose a new any-to-any voice conversion pipeline. Our approach uses automated speech recognition features, pitch tracking, and a state-of-the-art waveform prediction model.
  arXiv  Detail & Related papers  (2022-03-31T10:45:32Z)
• SIG-VC: A Speaker Information Guided Zero-shot Voice Conversion System for Both Human Beings and Machines [15.087294549955304]
  We aim to obtain intermediate representations for speaker-content disentanglement of speech. Speaker information control is added to our system to maintain the voice cloning performance. Results show that our proposed system significantly reduces the trade-off problem in zero-shot voice conversion.
  arXiv  Detail & Related papers  (2021-11-06T06:22:45Z)
• The Sequence-to-Sequence Baseline for the Voice Conversion Challenge 2020: Cascading ASR and TTS [66.06385966689965]
  This paper presents the sequence-to-sequence (seq2seq) baseline system for the voice conversion challenge (VCC) 2020. We consider a naive approach for voice conversion (VC), which is to first transcribe the input speech with an automatic speech recognition (ASR) model. We revisit this method under a sequence-to-sequence (seq2seq) framework by utilizing ESPnet, an open-source end-to-end speech processing toolkit.
  arXiv  Detail & Related papers  (2020-10-06T02:27:38Z)
• Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modeling [61.351967629600594]
  This paper proposes an any-to-many location-relative, sequence-to-sequence (seq2seq), non-parallel voice conversion approach. In this approach, we combine a bottle-neck feature extractor (BNE) with a seq2seq synthesis module. Objective and subjective evaluations show that the proposed any-to-many approach has superior voice conversion performance in terms of both naturalness and speaker similarity.
  arXiv  Detail & Related papers  (2020-09-06T13:01:06Z)
• Audio ALBERT: A Lite BERT for Self-supervised Learning of Audio Representation [51.37980448183019]
  We propose Audio ALBERT, a lite version of the self-supervised speech representation model. We show that Audio ALBERT is capable of achieving competitive performance with those huge models in the downstream tasks. In probing experiments, we find that the latent representations encode richer information of both phoneme and speaker than that of the last layer.
  arXiv  Detail & Related papers  (2020-05-18T10:42:44Z)
• Many-to-Many Voice Transformer Network [55.17770019619078]
  This paper proposes a voice conversion (VC) method based on a sequence-to-sequence (S2S) learning framework. It enables simultaneous conversion of the voice characteristics, pitch contour, and duration of input speech.
  arXiv  Detail & Related papers  (2020-05-18T04:02:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.