Reimagining Speech: A Scoping Review of Deep Learning-Powered Voice Conversion

• URL: http://arxiv.org/abs/2311.08104v1
• Date: Tue, 14 Nov 2023 12:03:46 GMT
• Title: Reimagining Speech: A Scoping Review of Deep Learning-Powered Voice Conversion
• Abstract summary: This paper presents a scoping review that explores the use of deep learning in speech analysis, synthesis, and disentangled speech representation learning within modern voice conversion systems. We screened 621 publications from more than 38 different venues between the years 2017 and 2023, followed by an in-depth review of a final database consisting of 123 eligible studies. We summarise the most frequently used approaches to voice conversion based on deep learning and highlight common pitfalls within the community.
• Score: 2.8034840754506214
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Research on deep learning-powered voice conversion (VC) in speech-to-speech scenarios is getting increasingly popular. Although many of the works in the field of voice conversion share a common global pipeline, there is a considerable diversity in the underlying structures, methods, and neural sub-blocks used across research efforts. Thus, obtaining a comprehensive understanding of the reasons behind the choice of the different methods in the voice conversion pipeline can be challenging, and the actual hurdles in the proposed solutions are often unclear. To shed light on these aspects, this paper presents a scoping review that explores the use of deep learning in speech analysis, synthesis, and disentangled speech representation learning within modern voice conversion systems. We screened 621 publications from more than 38 different venues between the years 2017 and 2023, followed by an in-depth review of a final database consisting of 123 eligible studies. Based on the review, we summarise the most frequently used approaches to voice conversion based on deep learning and highlight common pitfalls within the community. Lastly, we condense the knowledge gathered, identify main challenges and provide recommendations for future research directions.

Related papers

• Classification of Spontaneous and Scripted Speech for Multilingual Audio [9.925703861731506]
  Distinguishing scripted from spontaneous speech is an essential tool for better understanding how speech styles influence speech processing research. This paper addresses the challenge of building a classifier that generalises well across different formats and languages. We systematically evaluate models ranging from traditional, handcrafted acoustic and prosodic features to advanced audio transformers.
  arXiv  Detail & Related papers  (2024-12-16T15:45:10Z)
• Do Audio-Language Models Understand Linguistic Variations? [42.17718387132912]
  Open-vocabulary audio language models (ALMs) represent a promising new paradigm for audio-text retrieval using natural language queries. We propose RobustCLAP, a novel and compute-efficient technique to learn audio-language representations to linguistic variations.
  arXiv  Detail & Related papers  (2024-10-21T20:55:33Z)
• Towards Investigating Biases in Spoken Conversational Search [10.120634413661929]
  We review how biases and user attitude changes have been studied in screen-based web search. We propose an experimental setup with variables, data, and instruments to explore biases in a voice-based setting like Spoken Conversational Search.
  arXiv  Detail & Related papers  (2024-09-02T01:54:33Z)
• Transformers in Speech Processing: A Survey [4.984401393225283]
  transformers have gained prominence across various speech-related domains, including automatic speech recognition, speech synthesis, speech translation, speech para-linguistics, speech enhancement, spoken dialogue systems, and multimodal applications. We present a comprehensive survey that aims to bridge research studies from diverse subfields within speech technology.
  arXiv  Detail & Related papers  (2023-03-21T06:00:39Z)
• Deep Learning for Visual Speech Analysis: A Survey [54.53032361204449]
  This paper presents a review of recent progress in deep learning methods on visual speech analysis. We cover different aspects of visual speech, including fundamental problems, challenges, benchmark datasets, a taxonomy of existing methods, and state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-05-22T14:44:53Z)
• Automated Audio Captioning: an Overview of Recent Progress and New Challenges [56.98522404673527]
  Automated audio captioning is a cross-modal translation task that aims to generate natural language descriptions for given audio clips. We present a comprehensive review of the published contributions in automated audio captioning, from a variety of existing approaches to evaluation metrics and datasets.
  arXiv  Detail & Related papers  (2022-05-12T08:36:35Z)
• Spoken Style Learning with Multi-modal Hierarchical Context Encoding for Conversational Text-to-Speech Synthesis [59.27994987902646]
  The study about learning spoken styles from historical conversations is still in its infancy. Only the transcripts of the historical conversations are considered, which neglects the spoken styles in historical speeches. We propose a spoken style learning approach with multi-modal hierarchical context encoding.
  arXiv  Detail & Related papers  (2021-06-11T08:33:52Z)
• Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapters [52.725200145600624]
  We propose KnowExpert to bypass the retrieval process by injecting prior knowledge into the pre-trained language models with lightweight adapters. Experimental results show that KnowExpert performs comparably with the retrieval-based baselines.
  arXiv  Detail & Related papers  (2021-05-13T12:33:23Z)
• An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation [57.68765353264689]
  Speech enhancement and speech separation are two related tasks. Traditionally, these tasks have been tackled using signal processing and machine learning techniques. Deep learning has been exploited to achieve strong performance.
  arXiv  Detail & Related papers  (2020-08-21T17:24:09Z)
• Deep Audio-Visual Learning: A Survey [53.487938108404244]
  We divide the current audio-visual learning tasks into four different subfields. We discuss state-of-the-art methods as well as the remaining challenges of each subfield. We summarize the commonly used datasets and performance metrics.
  arXiv  Detail & Related papers  (2020-01-14T13:11:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.