GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for All Singing Tasks

• URL: http://arxiv.org/abs/2409.13832v4
• Date: Wed, 30 Oct 2024 04:37:33 GMT
• Title: GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for All Singing Tasks
• Authors: Yu Zhang, Changhao Pan, Wenxiang Guo, Ruiqi Li, Zhiyuan Zhu, Jialei Wang, Wenhao Xu, Jingyu Lu, Zhiqing Hong, Chuxin Wang, LiChao Zhang, Jinzheng He, Ziyue Jiang, Yuxin Chen, Chen Yang, Jiecheng Zhou, Xinyu Cheng, Zhou Zhao,
• Abstract summary: GTSinger is a large global, multi-technique, free-to-use, high-quality singing corpus with realistic music scores. We collect 80.59 hours of high-quality singing voices, forming the largest recorded singing dataset. We conduct four benchmark experiments: technique-controllable singing voice synthesis, technique recognition, style transfer, and speech-to-singing conversion.
• Score: 52.30565320125514
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The scarcity of high-quality and multi-task singing datasets significantly hinders the development of diverse controllable and personalized singing tasks, as existing singing datasets suffer from low quality, limited diversity of languages and singers, absence of multi-technique information and realistic music scores, and poor task suitability. To tackle these problems, we present GTSinger, a large global, multi-technique, free-to-use, high-quality singing corpus with realistic music scores, designed for all singing tasks, along with its benchmarks. Particularly, (1) we collect 80.59 hours of high-quality singing voices, forming the largest recorded singing dataset; (2) 20 professional singers across nine widely spoken languages offer diverse timbres and styles; (3) we provide controlled comparison and phoneme-level annotations of six commonly used singing techniques, helping technique modeling and control; (4) GTSinger offers realistic music scores, assisting real-world musical composition; (5) singing voices are accompanied by manual phoneme-to-audio alignments, global style labels, and 16.16 hours of paired speech for various singing tasks. Moreover, to facilitate the use of GTSinger, we conduct four benchmark experiments: technique-controllable singing voice synthesis, technique recognition, style transfer, and speech-to-singing conversion. The corpus and demos can be found at http://gtsinger.github.io. We provide the dataset and the code for processing data and conducting benchmarks at https://huggingface.co/datasets/GTSinger/GTSinger and https://github.com/GTSinger/GTSinger.

Related papers

• TCSinger: Zero-Shot Singing Voice Synthesis with Style Transfer and Multi-Level Style Control [58.96445085236971]
  Zero-shot singing voice synthesis (SVS) with style transfer and style control aims to generate high-quality singing voices with unseen timbres and styles. We introduce TCSinger, the first zero-shot SVS model for style transfer across cross-lingual speech and singing styles. We show that TCSinger outperforms all baseline models in quality synthesis, singer similarity, and style controllability across various tasks.
  arXiv  Detail & Related papers  (2024-09-24T11:18:09Z)
• Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt [50.25271407721519]
  We propose Prompt-Singer, the first SVS method that enables attribute controlling on singer gender, vocal range and volume with natural language. We adopt a model architecture based on a decoder-only transformer with a multi-scale hierarchy, and design a range-melody decoupled pitch representation. Experiments show that our model achieves favorable controlling ability and audio quality.
  arXiv  Detail & Related papers  (2024-03-18T13:39:05Z)
• Singer Identity Representation Learning using Self-Supervised Techniques [0.0]
  We propose a framework for training singer identity encoders to extract representations suitable for various singing-related tasks. We explore different self-supervised learning techniques on a large collection of isolated vocal tracks. We evaluate the quality of the resulting representations on singer similarity and identification tasks.
  arXiv  Detail & Related papers  (2024-01-10T10:41:38Z)
• SingingHead: A Large-scale 4D Dataset for Singing Head Animation [75.63669264992134]
  We collect a large-scale singing head dataset, SingingHead, which consists of more than 27 hours of synchronized singing video, 3D facial motion, singing audio, and background music. Along with the SingingHead dataset, we benchmark existing audio-driven 3D facial animation methods and 2D talking head methods on the singing task. We propose a unified singing head animation framework named UniSinger to achieve both singing audio-driven 3D singing head animation and 2D singing portrait video synthesis.
  arXiv  Detail & Related papers  (2023-12-07T15:40:36Z)
• Learning the Beauty in Songs: Neural Singing Voice Beautifier [69.21263011242907]
  We are interested in a novel task, singing voice beautifying (SVB) Given the singing voice of an amateur singer, SVB aims to improve the intonation and vocal tone of the voice, while keeping the content and vocal timbre. We introduce Neural Singing Voice Beautifier (NSVB), the first generative model to solve the SVB task.
  arXiv  Detail & Related papers  (2022-02-27T03:10:12Z)
• Deep Learning Approach for Singer Voice Classification of Vietnamese Popular Music [1.2043574473965315]
  We propose a new method to identify the singer's name based on analysis of Vietnamese popular music. We employ the use of vocal segment detection and singing voice separation as the pre-processing steps. To verify the accuracy of our methods, we evaluate on a dataset of 300 Vietnamese songs from 18 famous singers.
  arXiv  Detail & Related papers  (2021-02-24T08:03:07Z)
• DeepSinger: Singing Voice Synthesis with Data Mined From the Web [194.10598657846145]
  DeepSinger is a multi-lingual singing voice synthesis system built from scratch using singing training data mined from music websites. We evaluate DeepSinger on our mined singing dataset that consists of about 92 hours data from 89 singers on three languages.
  arXiv  Detail & Related papers  (2020-07-09T07:00:48Z)
• Adversarially Trained Multi-Singer Sequence-To-Sequence Singing Synthesizer [11.598416444452619]
  We design a multi-singer framework to leverage all the existing singing data of different singers. We incorporate an adversarial task of singer classification to make encoder output less singer dependent. The proposed synthesizer can generate higher quality singing voice than baseline.
  arXiv  Detail & Related papers  (2020-06-18T07:20:11Z)
• Addressing the confounds of accompaniments in singer identification [29.949390919663596]
  We employ open-unmix, an open source tool with state-of-the-art performance in source separation, to separate the vocal and instrumental tracks of music. We then investigate two means to train a singer identification model: by learning from the separated vocal only, or from an augmented set of data.
  arXiv  Detail & Related papers  (2020-02-17T07:49:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.