Wind Noise Reduction with a Diffusion-based Stochastic Regeneration Model

• URL: http://arxiv.org/abs/2306.12867v2
• Date: Tue, 9 Jan 2024 08:46:16 GMT
• Title: Wind Noise Reduction with a Diffusion-based Stochastic Regeneration Model
• Authors: Jean-Marie Lemercier, Joachim Thiemann, Raphael Koning, Timo Gerkmann
• Abstract summary: We present a method for single-channel wind noise reduction using our previously proposed diffusion-based regeneration model. We introduce a non-additive speech in noise model to account for the non-linear deformation of the membrane caused by the wind flow and possible clipping.
• Score: 19.156383933702884
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper we present a method for single-channel wind noise reduction using our previously proposed diffusion-based stochastic regeneration model combining predictive and generative modelling. We introduce a non-additive speech in noise model to account for the non-linear deformation of the membrane caused by the wind flow and possible clipping. We show that our stochastic regeneration model outperforms other neural-network-based wind noise reduction methods as well as purely predictive and generative models, on a dataset using simulated and real-recorded wind noise. We further show that the proposed method generalizes well by testing on an unseen dataset with real-recorded wind noise. Audio samples, data generation scripts and code for the proposed methods can be found online (https://uhh.de/inf-sp-storm-wind).

Related papers

• Rectified Noise: A Generative Model Using Positive-incentive Noise [9.097754636179902]
  Rectified Flow (RF) has been widely used as an effective generative model.<n>We propose an innovative generative algorithm to train pi-noise generators, namely Rectified Noise (RN)<n>RN improves the generative performance by injecting pi-noise into the velocity field of pre-trained RF models.
  arXiv  Detail & Related papers  (2025-11-11T07:08:35Z)
• Mitigating the Noise Shift for Denoising Generative Models via Noise Awareness Guidance [54.88271057438763]
  Noise Awareness Guidance (NAG) is a correction method that explicitly steers sampling trajectories to remain consistent with the pre-defined noise schedule.<n>NAG consistently mitigates noise shift and substantially improves the generation quality of mainstream diffusion models.
  arXiv  Detail & Related papers  (2025-10-14T13:31:34Z)
• On the Shape of Latent Variables in a Denoising VAE-MoG: A Posterior Sampling-Based Study [51.56484100374058]
  We explore the latent space of a denoising variational autoencoder with a mixture-of-Gaussians prior (VAE-MoG)<n>To evaluate how well the model captures the underlying structure, we use Hamiltonian Monte Carlo (HMC) to draw posterior samples conditioned on clean inputs, and compare them to the encoder's outputs from noisy data.<n>Although the model reconstructs signals accurately, statistical comparisons reveal a clear mismatch in the latent space.
  arXiv  Detail & Related papers  (2025-09-29T18:33:09Z)
• Noise Conditional Variational Score Distillation [60.38982038894823]
  Noise Conditional Variational Score Distillation (NCVSD) is a novel method for distilling pretrained diffusion models into generative denoisers.<n>By integrating this insight into the Variational Score Distillation framework, we enable scalable learning of generative denoisers.
  arXiv  Detail & Related papers  (2025-06-11T06:01:39Z)
• Alternators With Noise Models [2.992602379681373]
  This paper introduces a new model, called++, which enhances the flexibility of traditionalGrads by explicitly modeling the noise terms used to sample the latent and observed trajectories.<n>We demonstrate the effectiveness of++ in tasks such as density estimation, time series imputation, and forecasting, showing that it outperforms several strong baselines.
  arXiv  Detail & Related papers  (2025-05-18T21:01:45Z)
• Generative Modeling with Bayesian Sample Inference [50.07758840675341]
  We derive a novel generative model from the simple act of Gaussian posterior inference. Treating the generated sample as an unknown variable to infer lets us formulate the sampling process in the language of Bayesian probability. Our model uses a sequence of prediction and posterior update steps to narrow down the unknown sample from a broad initial belief.
  arXiv  Detail & Related papers  (2025-02-11T14:27:10Z)
• Diffusion-based Unsupervised Audio-visual Speech Enhancement [26.937216751657697]
  This paper proposes a new unsupervised audiovisual speech enhancement (AVSE) approach. It combines a diffusion-based audio-visual speech generative model with a non-negative matrix factorization (NMF) noise model. Experimental results confirm that the proposed AVSE approach not only outperforms its audio-only counterpart but also generalizes better than a recent supervisedgenerative AVSE method.
  arXiv  Detail & Related papers  (2024-10-04T12:22:54Z)
• DiffATR: Diffusion-based Generative Modeling for Audio-Text Retrieval [49.076590578101985]
  We present a diffusion-based ATR framework (DiffATR) that generates joint distribution from noise. Experiments on the AudioCaps and Clotho datasets with superior performances, verify the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-09-16T06:33:26Z)
• Diffusion Gaussian Mixture Audio Denoise [23.760755498636943]
  We propose a DiffGMM model, a denoising model based on the diffusion and Gaussian mixture models. Given a noisy audio signal, we first apply a 1D-U-Net to extract features and train linear layers to estimate parameters for the Gaussian mixture model. The noisy signal is continuously subtracted from the estimated noise to output clean audio signals.
  arXiv  Detail & Related papers  (2024-06-13T14:18:10Z)
• One Noise to Rule Them All: Learning a Unified Model of Spatially-Varying Noise Patterns [33.293193191683145]
  We present a single generative model which can learn to generate multiple types of noise as well as blend between them. We also present an application of our model to improving inverse procedural material design.
  arXiv  Detail & Related papers  (2024-04-25T02:23:11Z)
• Blue noise for diffusion models [50.99852321110366]
  We introduce a novel and general class of diffusion models taking correlated noise within and across images into account. Our framework allows introducing correlation across images within a single mini-batch to improve gradient flow. We perform both qualitative and quantitative evaluations on a variety of datasets using our method.
  arXiv  Detail & Related papers  (2024-02-07T14:59:25Z)
• Diffusion Posterior Sampling for Informed Single-Channel Dereverberation [15.16865739526702]
  We present an informed single-channel dereverberation method based on conditional generation with diffusion models. With knowledge of the room impulse response, the anechoic utterance is generated via reverse diffusion. The proposed approach is largely more robust to measurement noise compared to a state-of-the-art informed single-channel dereverberation method.
  arXiv  Detail & Related papers  (2023-06-21T14:14:05Z)
• To smooth a cloud or to pin it down: Guarantees and Insights on Score Matching in Denoising Diffusion Models [20.315727650065007]
  Denoising diffusion models are a class of generative models which have recently achieved state-of-the-art results across many domains. We leverage known connections to control akin to the F"ollmer drift to extend established neural network approximation results for the F"ollmer drift to denoising diffusion models and samplers.
  arXiv  Detail & Related papers  (2023-05-16T16:56:19Z)
• DiffusionAD: Norm-guided One-step Denoising Diffusion for Anomaly Detection [89.49600182243306]
  We reformulate the reconstruction process using a diffusion model into a noise-to-norm paradigm. We propose a rapid one-step denoising paradigm, significantly faster than the traditional iterative denoising in diffusion models. The segmentation sub-network predicts pixel-level anomaly scores using the input image and its anomaly-free restoration.
  arXiv  Detail & Related papers  (2023-03-15T16:14:06Z)
• Modeling Temporal Data as Continuous Functions with Stochastic Process Diffusion [2.2849153854336763]
  temporal data can be viewed as discretized measurements of the underlying function. To build a generative model for such data we have to model the process that governs it. We propose a solution by defining the denoising diffusion model in the function space.
  arXiv  Detail & Related papers  (2022-11-04T17:02:01Z)
• Conditional Diffusion Probabilistic Model for Speech Enhancement [101.4893074984667]
  We propose a novel speech enhancement algorithm that incorporates characteristics of the observed noisy speech signal into the diffusion and reverse processes. In our experiments, we demonstrate strong performance of the proposed approach compared to representative generative models.
  arXiv  Detail & Related papers  (2022-02-10T18:58:01Z)
• A Study on Speech Enhancement Based on Diffusion Probabilistic Model [63.38586161802788]
  We propose a diffusion probabilistic model-based speech enhancement model (DiffuSE) model that aims to recover clean speech signals from noisy signals. The experimental results show that DiffuSE yields performance that is comparable to related audio generative models on the standardized Voice Bank corpus task.
  arXiv  Detail & Related papers  (2021-07-25T19:23:18Z)
• Learning Noise-Aware Encoder-Decoder from Noisy Labels by Alternating Back-Propagation for Saliency Detection [54.98042023365694]
  We propose a noise-aware encoder-decoder framework to disentangle a clean saliency predictor from noisy training examples. The proposed model consists of two sub-models parameterized by neural networks.
  arXiv  Detail & Related papers  (2020-07-23T18:47:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.