Latent Nonlinear Denoising Score Matching for Enhanced Learning of Structured Distributions

• URL: http://arxiv.org/abs/2512.06615v1
• Date: Sun, 07 Dec 2025 01:17:14 GMT
• Title: Latent Nonlinear Denoising Score Matching for Enhanced Learning of Structured Distributions
• Authors: Kaichen Shen, Wei Zhu,
• Abstract summary: We present latent nonlinear denoising score matching (LNDSM)<n>LNDSM is a novel training objective for score-based generative models that integrates nonlinear forward dynamics with the VAE-based latent SGM framework.<n> Experiments on variants of the MNIST dataset demonstrate that the proposed method achieves faster synthesis and enhanced learning of inherently structured distributions.
• Score: 3.060720241524644
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present latent nonlinear denoising score matching (LNDSM), a novel training objective for score-based generative models that integrates nonlinear forward dynamics with the VAE-based latent SGM framework. This combination is achieved by reformulating the cross-entropy term using the approximate Gaussian transition induced by the Euler-Maruyama scheme. To ensure numerical stability, we identify and remove two zero-mean but variance exploding terms arising from small time steps. Experiments on variants of the MNIST dataset demonstrate that the proposed method achieves faster synthesis and enhanced learning of inherently structured distributions. Compared to benchmark structure-agnostic latent SGMs, LNDSM consistently attains superior sample quality and variability.

Related papers

• Inference-Time Scaling of Diffusion Language Models with Particle Gibbs Sampling [70.8832906871441]
  We study how to steer generation toward desired rewards without retraining the models.<n>Prior methods typically resample or filter within a single denoising trajectory, optimizing rewards step-by-step without trajectory-level refinement.<n>We introduce particle Gibbs sampling for diffusion language models (PG-DLM), a novel inference-time algorithm enabling trajectory-level refinement while preserving generation perplexity.
  arXiv  Detail & Related papers  (2025-07-11T08:00:47Z)
• Nonlinear Assimilation via Score-based Sequential Langevin Sampling [5.107329143106734]
  This paper presents score-based sequential Langevin sampling (SSLS)<n>The proposed method decomposes the assimilation process into alternating prediction and update steps.<n>We provide theoretical guarantees for SSLS convergence in total variation (TV) distance under certain conditions.
  arXiv  Detail & Related papers  (2024-11-20T16:31:46Z)
• Nonlinear denoising score matching for enhanced learning of structured distributions [12.428200977408817]
  Generalizing to a nonlinear drift allows for additional structure to be incorporated into the dynamics.<n>Our method learns score-based generative models with less data by flexibly incorporating structure arising in the dataset.
  arXiv  Detail & Related papers  (2024-05-24T15:14:23Z)
• Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference [47.460898983429374]
  We introduce an ensemble Kalman filter (EnKF) into the non-mean-field (NMF) variational inference framework to approximate the posterior distribution of the latent states. This novel marriage between EnKF and GPSSM not only eliminates the need for extensive parameterization in learning variational distributions, but also enables an interpretable, closed-form approximation of the evidence lower bound (ELBO) We demonstrate that the resulting EnKF-aided online algorithm embodies a principled objective function by ensuring data-fitting accuracy while incorporating model regularizations to mitigate overfitting.
  arXiv  Detail & Related papers  (2023-12-10T15:22:30Z)
• CoLiDE: Concomitant Linear DAG Estimation [12.415463205960156]
  We deal with the problem of learning acyclic graph structure from observational data to a linear equation. We propose a new convex score function for sparsity-aware learning DAGs.
  arXiv  Detail & Related papers  (2023-10-04T15:32:27Z)
• Regularizing with Pseudo-Negatives for Continual Self-Supervised Learning [62.40718385934608]
  We introduce a novel Pseudo-Negative Regularization (PNR) framework for effective continual self-supervised learning (CSSL) Our PNR leverages pseudo-negatives obtained through model-based augmentation in a way that newly learned representations may not contradict what has been learned in the past.
  arXiv  Detail & Related papers  (2023-06-08T10:59:35Z)
• Conditional Denoising Diffusion for Sequential Recommendation [62.127862728308045]
  Two prominent generative models, Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs) GANs suffer from unstable optimization, while VAEs are prone to posterior collapse and over-smoothed generations. We present a conditional denoising diffusion model, which includes a sequence encoder, a cross-attentive denoising decoder, and a step-wise diffuser.
  arXiv  Detail & Related papers  (2023-04-22T15:32:59Z)
• Leveraging Heteroscedastic Uncertainty in Learning Complex Spectral Mapping for Single-channel Speech Enhancement [20.823177372464414]
  Most speech enhancement (SE) models learn a point estimate, and do not make use of uncertainty estimation in the learning process. We show that modeling heteroscedastic uncertainty by minimizing a multivariate Gaussian negative log-likelihood (NLL) improves SE performance at no extra cost.
  arXiv  Detail & Related papers  (2022-11-16T02:29:05Z)
• Spectral Unmixing With Multinomial Mixture Kernel and Wasserstein Generative Adversarial Loss [4.56877715768796]
  This study proposes a novel framework for spectral unmixing by using 1D convolution kernels and spectral uncertainty. High-level representations are computed from data, and they are further modeled with the Multinomial Mixture Model. Experiments are performed on both real and synthetic datasets.
  arXiv  Detail & Related papers  (2020-12-12T16:49:01Z)
• Autoregressive Score Matching [113.4502004812927]
  We propose autoregressive conditional score models (AR-CSM) where we parameterize the joint distribution in terms of the derivatives of univariable log-conditionals (scores) For AR-CSM models, this divergence between data and model distributions can be computed and optimized efficiently, requiring no expensive sampling or adversarial training. We show with extensive experimental results that it can be applied to density estimation on synthetic data, image generation, image denoising, and training latent variable models with implicit encoders.
  arXiv  Detail & Related papers  (2020-10-24T07:01:24Z)
• Learning Mixtures of Low-Rank Models [89.39877968115833]
  We study the problem of learning computational mixtures of low-rank models. We develop an algorithm that is guaranteed to recover the unknown matrices with near-optimal sample. In addition, the proposed algorithm is provably stable against random noise.
  arXiv  Detail & Related papers  (2020-09-23T17:53:48Z)
• Adaptive Correlated Monte Carlo for Contextual Categorical Sequence Generation [77.7420231319632]
  We adapt contextual generation of categorical sequences to a policy gradient estimator, which evaluates a set of correlated Monte Carlo (MC) rollouts for variance control. We also demonstrate the use of correlated MC rollouts for binary-tree softmax models, which reduce the high generation cost in large vocabulary scenarios.
  arXiv  Detail & Related papers  (2019-12-31T03:01:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.