On the Collapse Errors Induced by the Deterministic Sampler for Diffusion Models

• URL: http://arxiv.org/abs/2508.16154v1
• Date: Fri, 22 Aug 2025 07:26:24 GMT
• Title: On the Collapse Errors Induced by the Deterministic Sampler for Diffusion Models
• Authors: Yi Zhang, Zhenyu Liao, Jingfeng Wu, Difan Zou,
• Abstract summary: Collapse errors are a previously unrecognized phenomenon in ODE-based diffusion sampling.<n>We observe a see-saw effect, where score learning in low noise regimes adversely impacts the one in high noise regimes.<n>This misfitting in high noise regimes, coupled with the dynamics of deterministic samplers, ultimately causes collapse errors.
• Score: 38.99546114710447
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the widespread adoption of deterministic samplers in diffusion models (DMs), their potential limitations remain largely unexplored. In this paper, we identify collapse errors, a previously unrecognized phenomenon in ODE-based diffusion sampling, where the sampled data is overly concentrated in local data space. To quantify this effect, we introduce a novel metric and demonstrate that collapse errors occur across a variety of settings. When investigating its underlying causes, we observe a see-saw effect, where score learning in low noise regimes adversely impacts the one in high noise regimes. This misfitting in high noise regimes, coupled with the dynamics of deterministic samplers, ultimately causes collapse errors. Guided by these insights, we apply existing techniques from sampling, training, and architecture to empirically support our explanation of collapse errors. This work provides intensive empirical evidence of collapse errors in ODE-based diffusion sampling, emphasizing the need for further research into the interplay between score learning and deterministic sampling, an overlooked yet fundamental aspect of diffusion models.

Related papers

• Score-based diffusion models for diffuse optical tomography with uncertainty quantification [0.8443238959374133]
  We introduce a novel regularization approach that prevents overfitting of the score function by constructing a mixed score composed of a learned and a model-based component.<n>Experiments demonstrate that a data-driven prior distribution results in posterior samples with low variance, compared to classical model-based estimation.
  arXiv  Detail & Related papers  (2026-02-03T12:14:07Z)
• Provable Maximum Entropy Manifold Exploration via Diffusion Models [58.89696361871563]
  Exploration is critical for solving real-world decision-making problems such as scientific discovery.<n>We introduce a novel framework that casts exploration as entropy over approximate data manifold implicitly defined by a pre-trained diffusion model.<n>We develop an algorithm based on mirror descent that solves the exploration problem as sequential fine-tuning of a pre-trained diffusion model.
  arXiv  Detail & Related papers  (2025-06-18T11:59:15Z)
• Diffusion models under low-noise regime [3.729242965449096]
  We show that diffusion models are effective denoisers when the corruption level is small.<n>We quantify how training set size, data geometry, and model objective choice shape denoising trajectories.<n>This work starts to address gaps in our understanding of generative model reliability in practical applications.
  arXiv  Detail & Related papers  (2025-06-09T15:07:16Z)
• Generalized Diffusion Model with Adjusted Offset Noise [1.7767466724342067]
  We propose a generalized diffusion model that naturally incorporates additional noise within a rigorous probabilistic framework.<n>We derive a loss function based on the evidence lower bound, establishing its theoretical equivalence to offset noise with certain adjustments.<n>Experiments on synthetic datasets demonstrate that our model effectively addresses brightness-related challenges and outperforms conventional methods in high-dimensional scenarios.
  arXiv  Detail & Related papers  (2024-12-04T08:57:03Z)
• Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data [74.2507346810066]
  Ambient diffusion is a recently proposed framework for training diffusion models using corrupted data. We present the first framework for training diffusion models that provably sample from the uncorrupted distribution given only noisy training data.
  arXiv  Detail & Related papers  (2024-03-20T14:22:12Z)
• Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimation [53.27596811146316]
  Diffusion models operate over a sequence of timesteps instead of instantaneous input-output relationships in previous contexts. We present Diffusion-TracIn that incorporates this temporal dynamics and observe that samples' loss gradient norms are highly dependent on timestep. We introduce Diffusion-ReTrac as a re-normalized adaptation that enables the retrieval of training samples more targeted to the test sample of interest.
  arXiv  Detail & Related papers  (2024-01-17T07:58:18Z)
• Diffusion-C: Unveiling the Generative Challenges of Diffusion Models through Corrupted Data [2.7624021966289605]
  "Diffusion-C" is a foundational methodology to analyze the generative restrictions of Diffusion Models. Within the milieu of generative models under the Diffusion taxonomy, DDPM emerges as a paragon, consistently exhibiting superior performance metrics. The vulnerability of Diffusion Models to these particular corruptions is significantly influenced by topological and statistical similarities.
  arXiv  Detail & Related papers  (2023-12-14T12:01:51Z)
• Projection Regret: Reducing Background Bias for Novelty Detection via Diffusion Models [72.07462371883501]
  We propose emphProjection Regret (PR), an efficient novelty detection method that mitigates the bias of non-semantic information. PR computes the perceptual distance between the test image and its diffusion-based projection to detect abnormality. Extensive experiments demonstrate that PR outperforms the prior art of generative-model-based novelty detection methods by a significant margin.
  arXiv  Detail & Related papers  (2023-12-05T09:44:47Z)
• Elucidating the Exposure Bias in Diffusion Models [7.529885833916501]
  We investigate the input mismatch between training and sampling in diffusion models. We propose a training-free method called Epsilon Scaling to alleviate the exposure bias. We show that Epsilon Scaling explicitly moves the sampling trajectory closer to the vector field learned in the training phase.
  arXiv  Detail & Related papers  (2023-08-29T14:16:09Z)
• On Error Propagation of Diffusion Models [77.91480554418048]
  We develop a theoretical framework to mathematically formulate error propagation in the architecture of DMs. We apply the cumulative error as a regularization term to reduce error propagation. Our proposed regularization reduces error propagation, significantly improves vanilla DMs, and outperforms previous baselines.
  arXiv  Detail & Related papers  (2023-08-09T15:31:17Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.