WeSinger: Data-augmented Singing Voice Synthesis with Auxiliary Losses
- URL: http://arxiv.org/abs/2203.10750v2
- Date: Thu, 24 Mar 2022 03:57:17 GMT
- Title: WeSinger: Data-augmented Singing Voice Synthesis with Auxiliary Losses
- Authors: Zewang Zhang, Yibin Zheng, Xinhui Li, Li Lu
- Abstract summary: We develop a new multi-singer Chinese neural singing voice synthesis system named WeSinger.
quantitative and qualitative evaluation results demonstrate the effectiveness of WeSinger in terms of accuracy and naturalness.
- Score: 13.178747366560534
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In this paper, we develop a new multi-singer Chinese neural singing voice
synthesis (SVS) system named WeSinger. To improve the accuracy and naturalness
of synthesized singing voice, we design several specifical modules and
techniques: 1) A deep bi-directional LSTM based duration model with multi-scale
rhythm loss and post-processing step; 2) A Transformer-alike acoustic model
with progressive pitch-weighted decoder loss; 3) a 24 kHz pitch-aware LPCNet
neural vocoder to produce high-quality singing waveforms; 4) A novel data
augmentation method with multi-singer pre-training for stronger robustness and
naturalness. Both quantitative and qualitative evaluation results demonstrate
the effectiveness of WeSinger in terms of accuracy and naturalness, and
WeSinger achieves state-of-the-art performance on the public corpus Opencpop.
Some synthesized singing samples are available online
(https://zzw922cn.github.io/WeSinger/).
Related papers
- ConSinger: Efficient High-Fidelity Singing Voice Generation with Minimal Steps [4.319804315515349]
We propose a singing voice synthesis method based on the consistency model, ConSinger, to achieve high-fidelity singing voice synthesis with minimal steps.
Our experiments show that ConSinger is highly competitive with the baseline model in terms of generation speed and quality.
arXiv Detail & Related papers (2024-10-20T09:32:03Z) - Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt [50.25271407721519]
We propose Prompt-Singer, the first SVS method that enables attribute controlling on singer gender, vocal range and volume with natural language.
We adopt a model architecture based on a decoder-only transformer with a multi-scale hierarchy, and design a range-melody decoupled pitch representation.
Experiments show that our model achieves favorable controlling ability and audio quality.
arXiv Detail & Related papers (2024-03-18T13:39:05Z) - Enhancing the vocal range of single-speaker singing voice synthesis with
melody-unsupervised pre-training [82.94349771571642]
This work proposes a melody-unsupervised multi-speaker pre-training method to enhance the vocal range of the single-speaker.
It is the first to introduce a differentiable duration regulator to improve the rhythm naturalness of the synthesized voice.
Experimental results verify that the proposed SVS system outperforms the baseline on both sound quality and naturalness.
arXiv Detail & Related papers (2023-09-01T06:40:41Z) - Towards Improving the Expressiveness of Singing Voice Synthesis with
BERT Derived Semantic Information [51.02264447897833]
This paper presents an end-to-end high-quality singing voice synthesis (SVS) system that uses bidirectional encoder representation from Transformers (BERT) derived semantic embeddings.
The proposed SVS system can produce singing voice with higher-quality outperforming VISinger.
arXiv Detail & Related papers (2023-08-31T16:12:01Z) - NeuralDPS: Neural Deterministic Plus Stochastic Model with Multiband
Excitation for Noise-Controllable Waveform Generation [67.96138567288197]
We propose a novel neural vocoder named NeuralDPS which can retain high speech quality and acquire high synthesis efficiency and noise controllability.
It generates waveforms at least 280 times faster than the WaveNet vocoder.
It is also 28% faster than WaveGAN's synthesis efficiency on a single core.
arXiv Detail & Related papers (2022-03-05T08:15:29Z) - Sinsy: A Deep Neural Network-Based Singing Voice Synthesis System [25.573552964889963]
This paper presents Sinsy, a deep neural network (DNN)-based singing voice synthesis (SVS) system.
The proposed system is composed of four modules: a time-lag model, a duration model, an acoustic model, and a vocoder.
Experimental results show our system can synthesize a singing voice with better timing, more natural vibrato, and correct pitch.
arXiv Detail & Related papers (2021-08-05T17:59:58Z) - DiffSinger: Diffusion Acoustic Model for Singing Voice Synthesis [53.19363127760314]
DiffSinger is a parameterized Markov chain which iteratively converts the noise into mel-spectrogram conditioned on the music score.
The evaluations conducted on the Chinese singing dataset demonstrate that DiffSinger outperforms state-of-the-art SVS work with a notable margin.
arXiv Detail & Related papers (2021-05-06T05:21:42Z) - DeepSinger: Singing Voice Synthesis with Data Mined From the Web [194.10598657846145]
DeepSinger is a multi-lingual singing voice synthesis system built from scratch using singing training data mined from music websites.
We evaluate DeepSinger on our mined singing dataset that consists of about 92 hours data from 89 singers on three languages.
arXiv Detail & Related papers (2020-07-09T07:00:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.