The Loss Floor of Denoising Score Matching: Fisher Geometry from Schrödinger Bridges

• URL: http://arxiv.org/abs/2608.23916v1
• Date: Mon, 24 Aug 2026 23:48:06 GMT
• Title: The Loss Floor of Denoising Score Matching: Fisher Geometry from Schrödinger Bridges
• Abstract summary: We show that for a general corruption kernel under mild regularity assumptions, it is exactly the trace of the Fisher--Rao metric of the conditional endpoint family.<n>We derive the result from a Schr"odinger bridge variational principle, in which the ideal objective arises as excess path-space relative entropy.
• Score: 4.661816556403101
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Denoising score matching trains diffusion models by regressing onto a conditional score, although generation ultimately requires the marginal score. The two objectives share the same population minimizer, but the conditional target remains random at fixed noisy state and introduces an irreducible excess in the training loss. We isolate this excess and show that, for a general corruption kernel under mild regularity assumptions, it is exactly the trace of the Fisher--Rao metric of the conditional endpoint family, integrated along the diffusion trajectory. This gives an exact conditional-variance decomposition of the denoising objective and identifies the information geometry observed in diffusion latent spaces as an intrinsic component of the training loss. We derive the result from a Schr"odinger bridge variational principle, in which the ideal objective arises as excess path-space relative entropy. For corruption diffusions, the Fisher term is proportional to the rate at which the noisy state loses mutual information about the clean data, separating the loss floor into an information flow determined by the data and a weight determined by the corruption schedule and objective. In the Gaussian case, this yields a closed form for the floor, recovers reparametrization invariance of the continuous-time objective, and relates its high-SNR divergence to the information dimension of the data. Finally, we show that raw losses obtained with different noise ranges or weightings need not rank models consistently because they contain different additive floors, and contrast the second-order geometry seen by training with the third-order conditional statistics entering numerical sampling error.

Related papers

• Distributionally Faithful Imputation via Positive Semi-Definite Kernel Density Estimation [55.47411235954028]
  We recast imputation under missing completely at random as density estimation from masked observations.<n>We obtain a convex empirical risk problem with closed form marginals, solvable by a Newton interior point method.<n>The resulting PSD Impute model yields both single and multiple imputations from the same fitted density.
  arXiv  Detail & Related papers  (2026-07-08T15:51:05Z)
• Density-Ratio Losses for Post-Hoc Learning to Defer [61.81110533670321]
  We study post-hoc Learning to Defer (L2D) through the lens of ideal distributions.<n>We define deferral via the density-ratio between a model's and an expert's ideals.<n>Deferral decisions are then made by thresholding the scorer, allowing deferral rates to be adjusted without retraining.
  arXiv  Detail & Related papers  (2026-05-19T08:56:23Z)
• Conditional Diffusion Under Linear Constraints: Langevin Mixing and Information-Theoretic Guarantees [10.73598638822051]
  We study zero-shot conditional sampling with pretrained diffusion models for linear inverse problems.<n>We prove that the error from replacing this score by the unconditional tangent score is upper bounded by a dimension-free conditional mutual information between observed and unobserved components.
  arXiv  Detail & Related papers  (2026-05-06T19:19:54Z)
• Generative Modeling under Non-Monotonic MAR Missingness via Approximate Wasserstein Gradient Flows [6.221876252052772]
  We propose a principled iterative method for generating a complete dataset from a dataset with values Missing at Random (MAR)<n>FLOWGEM minimizes the expected Kullback-Leibler (KL) divergence between the observed data distribution and the distribution of the generated sample over different missingness patterns.<n>It achieves state-of-the-art performance across a range of settings, including the challenging case of non-monotonic MAR mechanisms.
  arXiv  Detail & Related papers  (2026-04-06T09:56:08Z)
• A Unified View of Drifting and Score-Based Models [92.51878018325031]
  We make our relationship to the score-matching principle behind diffusion models precise by showing that drifting admits a score-based formulation on kernel-smoothed distributions.<n>We derive an exact decomposition for general radial kernels, and for the Laplace kernel we prove rigorous error bounds showing that drifting remains an accurate proxy for score matching in low-temperature and high-dimensional regimes.
  arXiv  Detail & Related papers  (2026-03-08T07:41:36Z)
• Information-Theoretic Discrete Diffusion [8.018632880023336]
  We present an information-theoretic framework for discrete diffusion models that yields principled estimators of log-likelihood using score-matching losses.<n>Results provide a time-integral decomposition of the log-likelihood of the data in terms of optimal score-based losses.<n>Experiments on synthetic and real-world data confirm the accuracy, variance stability, and utility of our estimators.
  arXiv  Detail & Related papers  (2025-10-28T05:59:05Z)
• Kernel-Smoothed Scores for Denoising Diffusion: A Bias-Variance Study [3.265950484493743]
  Diffusion models can be prone to memorization.<n>Regularization on the score has the same effect as increasing the size of the training dataset.<n>This perspective highlights two regularization mechanisms taking place in denoising diffusions.
  arXiv  Detail & Related papers  (2025-05-28T20:22:18Z)
• Rejection via Learning Density Ratios [50.91522897152437]
  Classification with rejection emerges as a learning paradigm which allows models to abstain from making predictions.<n>We propose a different distributional perspective, where we seek to find an idealized data distribution which maximizes a pretrained model's performance.<n>Our framework is tested empirically over clean and noisy datasets.
  arXiv  Detail & Related papers  (2024-05-29T01:32:17Z)
• Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistent [97.64313409741614]
  We propose to enforce a emphconsistency property which states that predictions of the model on its own generated data are consistent across time. We show that our novel training objective yields state-of-the-art results for conditional and unconditional generation in CIFAR-10 and baseline improvements in AFHQ and FFHQ.
  arXiv  Detail & Related papers  (2023-02-17T18:45:04Z)
• FP-Diffusion: Improving Score-based Diffusion Models by Enforcing the Underlying Score Fokker-Planck Equation [72.19198763459448]
  We learn a family of noise-conditional score functions corresponding to the data density perturbed with increasingly large amounts of noise. These perturbed data densities are linked together by the Fokker-Planck equation (FPE), a partial differential equation (PDE) governing the spatial-temporal evolution of a density. We derive a corresponding equation called the score FPE that characterizes the noise-conditional scores of the perturbed data densities.
  arXiv  Detail & Related papers  (2022-10-09T16:27:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.