Song Data Cleansing for End-to-End Neural Singer Diarization Using Neural Analysis and Synthesis Framework

• URL: http://arxiv.org/abs/2406.16315v1
• Date: Mon, 24 Jun 2024 04:48:29 GMT
• Title: Song Data Cleansing for End-to-End Neural Singer Diarization Using Neural Analysis and Synthesis Framework
• Authors: Hokuto Munakata, Ryo Terashima, Yusuke Fujita,
• Abstract summary: Our proposed model converts song data with choral singing which is commonly contained in popular music. We exploit the pre-trained NANSY++ to convert choral singing into clean, non-overlapped audio. We experimentally evaluated the EEND model trained with a dataset using annotated popular duet songs.
• Score: 7.12217278294376
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a data cleansing method that utilizes a neural analysis and synthesis (NANSY++) framework to train an end-to-end neural diarization model (EEND) for singer diarization. Our proposed model converts song data with choral singing which is commonly contained in popular music and unsuitable for generating a simulated dataset to the solo singing data. This cleansing is based on NANSY++, which is a framework trained to reconstruct an input non-overlapped audio signal. We exploit the pre-trained NANSY++ to convert choral singing into clean, non-overlapped audio. This cleansing process mitigates the mislabeling of choral singing to solo singing and helps the effective training of EEND models even when the majority of available song data contains choral singing sections. We experimentally evaluated the EEND model trained with a dataset using our proposed method using annotated popular duet songs. As a result, our proposed method improved 14.8 points in diarization error rate.

Related papers

• Thutmose Tagger: Single-pass neural model for Inverse Text Normalization [76.87664008338317]
  Inverse text normalization (ITN) is an essential post-processing step in automatic speech recognition. We present a dataset preparation method based on the granular alignment of ITN examples. One-to-one correspondence between tags and input words improves the interpretability of the model's predictions.
  arXiv  Detail & Related papers  (2022-07-29T20:39:02Z)
• Learning the Beauty in Songs: Neural Singing Voice Beautifier [69.21263011242907]
  We are interested in a novel task, singing voice beautifying (SVB) Given the singing voice of an amateur singer, SVB aims to improve the intonation and vocal tone of the voice, while keeping the content and vocal timbre. We introduce Neural Singing Voice Beautifier (NSVB), the first generative model to solve the SVB task.
  arXiv  Detail & Related papers  (2022-02-27T03:10:12Z)
• Step-unrolled Denoising Autoencoders for Text Generation [17.015573262373742]
  We propose a new generative model of text, Step-unrolled Denoising Autoencoder (SUNDAE) SUNDAE is repeatedly applied on a sequence of tokens, starting from random inputs and improving them each time until convergence. We present a simple new improvement operator that converges in fewer iterations than diffusion methods.
  arXiv  Detail & Related papers  (2021-12-13T16:00:33Z)
• DiffSinger: Diffusion Acoustic Model for Singing Voice Synthesis [53.19363127760314]
  DiffSinger is a parameterized Markov chain which iteratively converts the noise into mel-spectrogram conditioned on the music score. The evaluations conducted on the Chinese singing dataset demonstrate that DiffSinger outperforms state-of-the-art SVS work with a notable margin.
  arXiv  Detail & Related papers  (2021-05-06T05:21:42Z)
• Anyone GAN Sing [0.0]
  We present a method to synthesize the singing voice of a person using a Convolutional Long Short-term Memory (ConvLSTM) based GAN. Our work is inspired by WGANSing by Chandna et al.
  arXiv  Detail & Related papers  (2021-02-22T14:30:58Z)
• Artificially Synthesising Data for Audio Classification and Segmentation to Improve Speech and Music Detection in Radio Broadcast [0.0]
  We present a novel procedure that artificially synthesises data that resembles radio signals. We trained a Convolutional Recurrent Neural Network (CRNN) on this synthesised data and outperformed state-of-the-art algorithms for music-speech detection.
  arXiv  Detail & Related papers  (2021-02-19T14:47:05Z)
• VAW-GAN for Singing Voice Conversion with Non-parallel Training Data [81.79070894458322]
  We propose a singing voice conversion framework based on VAW-GAN. We train an encoder to disentangle singer identity and singing prosody (F0) from phonetic content. By conditioning on singer identity and F0, the decoder generates output spectral features with unseen target singer identity.
  arXiv  Detail & Related papers  (2020-08-10T09:44:10Z)
• Unsupervised Cross-Domain Singing Voice Conversion [105.1021715879586]
  We present a wav-to-wav generative model for the task of singing voice conversion from any identity. Our method utilizes both an acoustic model, trained for the task of automatic speech recognition, together with melody extracted features to drive a waveform-based generator.
  arXiv  Detail & Related papers  (2020-08-06T18:29:11Z)
• Learning to Denoise Historical Music [30.165194151843835]
  We propose an audio-to-audio neural network model that learns to denoise old music recordings. The network is trained with both reconstruction and adversarial objectives on a noisy music dataset. Our results show that the proposed method is effective in removing noise, while preserving the quality and details of the original music.
  arXiv  Detail & Related papers  (2020-08-05T10:05:44Z)
• DeepSinger: Singing Voice Synthesis with Data Mined From the Web [194.10598657846145]
  DeepSinger is a multi-lingual singing voice synthesis system built from scratch using singing training data mined from music websites. We evaluate DeepSinger on our mined singing dataset that consists of about 92 hours data from 89 singers on three languages.
  arXiv  Detail & Related papers  (2020-07-09T07:00:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.