XiaoiceSing: A High-Quality and Integrated Singing Voice Synthesis System

• URL: http://arxiv.org/abs/2006.06261v1
• Date: Thu, 11 Jun 2020 09:09:59 GMT
• Title: XiaoiceSing: A High-Quality and Integrated Singing Voice Synthesis System
• Abstract summary: XiaoiceSing is a singing voice synthesis system which employs an integrated network for spectrum, F0 and duration modeling. Experiment results show that XiaoiceSing outperforms the baseline system of convolutional neural networks by 1.44 MOS on sound quality, 1.18 on pronunciation accuracy and 1.38 on naturalness.
• Score: 31.283081762312964
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents XiaoiceSing, a high-quality singing voice synthesis system which employs an integrated network for spectrum, F0 and duration modeling. We follow the main architecture of FastSpeech while proposing some singing-specific design: 1) Besides phoneme ID and position encoding, features from musical score (e.g.note pitch and length) are also added. 2) To attenuate off-key issues, we add a residual connection in F0 prediction. 3) In addition to the duration loss of each phoneme, the duration of all the phonemes in a musical note is accumulated to calculate the syllable duration loss for rhythm enhancement. Experiment results show that XiaoiceSing outperforms the baseline system of convolutional neural networks by 1.44 MOS on sound quality, 1.18 on pronunciation accuracy and 1.38 on naturalness respectively. In two A/B tests, the proposed F0 and duration modeling methods achieve 97.3% and 84.3% preference rate over baseline respectively, which demonstrates the overwhelming advantages of XiaoiceSing.

Related papers

• VocalRender: Score-Native Singing Voice Synthesis for Real-World Composition [45.70355425430011]
  We propose VocalRender, a score-native system that directly synthesizes singing from lyrics, pitches, symbolic note values, and tempo.<n>It uses an interleaved lyric--note representation and an autoregressive diffusion model to generate continuous acoustic latents while predicting the output length.<n>It achieves strong intelligibility, strong melody control, and high speaker similarity across both in-domain and out-of-domain benchmarks.
  arXiv  Detail & Related papers  (2026-07-30T07:04:46Z)
• FreyaTTS Technical Report [0.0]
  We introduce Freya-TTS, a compact, tokenizer-free, Turkish-first text-to-speaker model.<n>It operates in the frozen continuous latent space of AudioVAE2 (16 kHz encode, 48 kHz decode), allowing the model to focus its capacity on text-to-latent mapping.<n>It achieves a band-matched word error rate (WER) of 8.0% and character error rate (CER) of 3.0%, outperforming substantially larger open-source systems.
  arXiv  Detail & Related papers  (2026-07-10T15:36:30Z)
• WhisQ: Cross-Modal Representation Learning for Text-to-Music MOS Prediction [0.0]
  Mean Opinion Score (MOS) prediction for text to music systems requires evaluating both overall musical quality and text prompt alignment.<n>This paper introduces WhisQ, a multimodal architecture that addresses this dual-assessment challenge through sequence level co-attention and optimal transport regularization.
  arXiv  Detail & Related papers  (2025-06-06T09:12:51Z)
• DeepSound-V1: Start to Think Step-by-Step in the Audio Generation from Videos [4.452513686760606]
  We propose a framework for audio generation from videos, leveraging the internal chain-of-thought (CoT) of a multi-modal large language model (MLLM) A corresponding multi-modal reasoning dataset is constructed to facilitate the learning of initial reasoning in audio generation. In experiments, we demonstrate the effectiveness of the proposed framework in reducing misalignment (voice-over) in generated audio.
  arXiv  Detail & Related papers  (2025-03-28T07:56:19Z)
• Quartered Spectral Envelope and 1D-CNN-based Classification of Normally Phonated and Whispered Speech [0.0]
  The presence of pitch and pitch harmonics in normal speech, and its absence in whispered speech, is evident in the spectral envelope of the Fourier transform. We propose the use of one dimensional convolutional neural networks (1D-CNN) to capture these features. The system yields an accuracy of 99.31% when trained and tested on the wTIMIT dataset, and 100% on the CHAINS dataset.
  arXiv  Detail & Related papers  (2024-08-25T07:17:11Z)
• Multilingual Audio-Visual Speech Recognition with Hybrid CTC/RNN-T Fast Conformer [59.57249127943914]
  We present a multilingual Audio-Visual Speech Recognition model incorporating several enhancements to improve performance and audio noise robustness. We increase the amount of audio-visual training data for six distinct languages, generating automatic transcriptions of unlabelled multilingual datasets. Our proposed model achieves new state-of-the-art performance on the LRS3 dataset, reaching WER of 0.8%.
  arXiv  Detail & Related papers  (2024-03-14T01:16:32Z)
• MBTFNet: Multi-Band Temporal-Frequency Neural Network For Singing Voice Enhancement [8.782080886602145]
  We propose a novel temporal-frequency neural network (MBTFNet) for singing voice enhancement. MBTFNet removes background music, noise and even backing vocals from singing recordings. Experiments show that our proposed model significantly outperforms several state-of-the-art SE and MSS models.
  arXiv  Detail & Related papers  (2023-10-06T16:44:47Z)
• Enhancing the vocal range of single-speaker singing voice synthesis with melody-unsupervised pre-training [82.94349771571642]
  This work proposes a melody-unsupervised multi-speaker pre-training method to enhance the vocal range of the single-speaker. It is the first to introduce a differentiable duration regulator to improve the rhythm naturalness of the synthesized voice. Experimental results verify that the proposed SVS system outperforms the baseline on both sound quality and naturalness.
  arXiv  Detail & Related papers  (2023-09-01T06:40:41Z)
• Speech-enhanced and Noise-aware Networks for Robust Speech Recognition [25.279902171523233]
  A noise-aware training framework based on two cascaded neural structures is proposed to jointly optimize speech enhancement and speech recognition. The two proposed systems achieve word error rate (WER) of 3.90% and 3.55%, respectively, on the Aurora-4 task. Compared with the best existing systems that use bigram and trigram language models for decoding, the proposed CNN-TDNNF-based system achieves a relative WER reduction of 15.20% and 33.53%, respectively.
  arXiv  Detail & Related papers  (2022-03-25T15:04:51Z)
• WeSinger: Data-augmented Singing Voice Synthesis with Auxiliary Losses [13.178747366560534]
  We develop a new multi-singer Chinese neural singing voice synthesis system named WeSinger. quantitative and qualitative evaluation results demonstrate the effectiveness of WeSinger in terms of accuracy and naturalness.
  arXiv  Detail & Related papers  (2022-03-21T06:42:44Z)
• Singing-Tacotron: Global duration control attention and dynamic filter for End-to-end singing voice synthesis [67.96138567288197]
  This paper proposes an end-to-end singing voice synthesis framework, named Singing-Tacotron. The main difference between the proposed framework and Tacotron is that the speech can be controlled significantly by the musical score's duration information.
  arXiv  Detail & Related papers  (2022-02-16T07:35:17Z)
• DiffSinger: Diffusion Acoustic Model for Singing Voice Synthesis [53.19363127760314]
  DiffSinger is a parameterized Markov chain which iteratively converts the noise into mel-spectrogram conditioned on the music score. The evaluations conducted on the Chinese singing dataset demonstrate that DiffSinger outperforms state-of-the-art SVS work with a notable margin.
  arXiv  Detail & Related papers  (2021-05-06T05:21:42Z)
• HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesis [153.48507947322886]
  HiFiSinger is an SVS system towards high-fidelity singing voice. It consists of a FastSpeech based acoustic model and a Parallel WaveGAN based vocoder. Experiment results show that HiFiSinger synthesizes high-fidelity singing voices with much higher quality.
  arXiv  Detail & Related papers  (2020-09-03T16:31:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.