VaPar Synth -- A Variational Parametric Model for Audio Synthesis

• URL: http://arxiv.org/abs/2004.00001v1
• Date: Mon, 30 Mar 2020 16:05:47 GMT
• Title: VaPar Synth -- A Variational Parametric Model for Audio Synthesis
• Authors: Krishna Subramani, Preeti Rao, Alexandre D'Hooge
• Abstract summary: We present VaPar Synth - a Variational Parametric Synthesizer which utilizes a conditional variational autoencoder (CVAE) trained on a suitable parametric representation. We demonstrate our proposed model's capabilities via the reconstruction and generation of instrumental tones with flexible control over their pitch.
• Score: 78.3405844354125
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the advent of data-driven statistical modeling and abundant computing power, researchers are turning increasingly to deep learning for audio synthesis. These methods try to model audio signals directly in the time or frequency domain. In the interest of more flexible control over the generated sound, it could be more useful to work with a parametric representation of the signal which corresponds more directly to the musical attributes such as pitch, dynamics and timbre. We present VaPar Synth - a Variational Parametric Synthesizer which utilizes a conditional variational autoencoder (CVAE) trained on a suitable parametric representation. We demonstrate our proposed model's capabilities via the reconstruction and generation of instrumental tones with flexible control over their pitch.

Related papers

• Synthesizer Sound Matching Using Audio Spectrogram Transformers [2.5944208050492183]
  We introduce a synthesizer sound matching model based on the Audio Spectrogram Transformer. We show that this model can reconstruct parameters of samples generated from a set of 16 parameters. We also provide audio examples demonstrating the out-of-domain model performance in emulating vocal imitations.
  arXiv  Detail & Related papers  (2024-07-23T16:58:14Z)
• Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt [50.25271407721519]
  We propose Prompt-Singer, the first SVS method that enables attribute controlling on singer gender, vocal range and volume with natural language. We adopt a model architecture based on a decoder-only transformer with a multi-scale hierarchy, and design a range-melody decoupled pitch representation. Experiments show that our model achieves favorable controlling ability and audio quality.
  arXiv  Detail & Related papers  (2024-03-18T13:39:05Z)
• DiffMoog: a Differentiable Modular Synthesizer for Sound Matching [48.33168531500444]
  DiffMoog is a differentiable modular synthesizer with a comprehensive set of modules typically found in commercial instruments. Being differentiable, it allows integration into neural networks, enabling automated sound matching. We introduce an open-source platform that comprises DiffMoog and an end-to-end sound matching framework.
  arXiv  Detail & Related papers  (2024-01-23T08:59:21Z)
• Make-A-Voice: Unified Voice Synthesis With Discrete Representation [77.3998611565557]
  Make-A-Voice is a unified framework for synthesizing and manipulating voice signals from discrete representations. We show that Make-A-Voice exhibits superior audio quality and style similarity compared with competitive baseline models.
  arXiv  Detail & Related papers  (2023-05-30T17:59:26Z)
• Synthesizer Preset Interpolation using Transformer Auto-Encoders [4.213427823201119]
  We introduce a bimodal auto-encoder neural network, which simultaneously processes presets using multi-head attention blocks, and audio using convolutions. This model has been tested on a popular frequency modulation synthesizer with more than one hundred parameters. After training, the proposed model can be integrated into commercial synthesizers for live or sound design tasks.
  arXiv  Detail & Related papers  (2022-10-27T15:20:18Z)
• DDX7: Differentiable FM Synthesis of Musical Instrument Sounds [7.829520196474829]
  Differentiable Digital Signal Processing (DDSP) has enabled nuanced audio rendering by Deep Neural Networks (DNNs) We present Differentiable DX7 (DDX7), a lightweight architecture for neural FM resynthesis of musical instrument sounds.
  arXiv  Detail & Related papers  (2022-08-12T08:39:45Z)
• Differentiable Digital Signal Processing Mixture Model for Synthesis Parameter Extraction from Mixture of Harmonic Sounds [29.012177604120048]
  A differentiable digital signal processing (DDSP) autoencoder is a musical sound that combines a deep neural network (DNN) and spectral modeling synthesis. It allows us to flexibly edit sounds by changing the fundamental frequency, timbre feature, and loudness (synthesis parameters) extracted from an input sound. It is designed for a monophonic harmonic sound and cannot handle mixtures of sounds harmonic.
  arXiv  Detail & Related papers  (2022-02-01T03:38:49Z)
• RAVE: A variational autoencoder for fast and high-quality neural audio synthesis [2.28438857884398]
  We introduce a Realtime Audio Variational autoEncoder (RAVE) allowing both fast and high-quality audio waveform synthesis. We show that our model is the first able to generate 48kHz audio signals, while simultaneously running 20 times faster than real-time on a standard laptop CPU.
  arXiv  Detail & Related papers  (2021-11-09T09:07:30Z)
• DiffSinger: Diffusion Acoustic Model for Singing Voice Synthesis [53.19363127760314]
  DiffSinger is a parameterized Markov chain which iteratively converts the noise into mel-spectrogram conditioned on the music score. The evaluations conducted on the Chinese singing dataset demonstrate that DiffSinger outperforms state-of-the-art SVS work with a notable margin.
  arXiv  Detail & Related papers  (2021-05-06T05:21:42Z)
• Synthesizer: Rethinking Self-Attention in Transformer Models [93.08171885200922]
  dot product self-attention is central and indispensable to state-of-the-art Transformer models. This paper investigates the true importance and contribution of the dot product-based self-attention mechanism on the performance of Transformer models.
  arXiv  Detail & Related papers  (2020-05-02T08:16:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.