Robust Learning of Diffusion Models with Extremely Noisy Conditions

• URL: http://arxiv.org/abs/2510.10149v1
• Date: Sat, 11 Oct 2025 10:16:15 GMT
• Title: Robust Learning of Diffusion Models with Extremely Noisy Conditions
• Authors: Xin Chen, Gillian Dobbie, Xinyu Wang, Feng Liu, Di Wang, Jingfeng Zhang,
• Abstract summary: Conditional diffusion models have the generative controllability by incorporating external conditions.<n>This paper introduces a robust learning framework to address extremely noisy conditions in conditional diffusion models.<n>Our approach achieves state-of-the-art performance across a range of noise levels on both class-conditional image generation and visuomotor policy generation tasks.
• Score: 28.519168402002293
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conditional diffusion models have the generative controllability by incorporating external conditions. However, their performance significantly degrades with noisy conditions, such as corrupted labels in the image generation or unreliable observations or states in the control policy generation. This paper introduces a robust learning framework to address extremely noisy conditions in conditional diffusion models. We empirically demonstrate that existing noise-robust methods fail when the noise level is high. To overcome this, we propose learning pseudo conditions as surrogates for clean conditions and refining pseudo ones progressively via the technique of temporal ensembling. Additionally, we develop a Reverse-time Diffusion Condition (RDC) technique, which diffuses pseudo conditions to reinforce the memorization effect and further facilitate the refinement of the pseudo conditions. Experimentally, our approach achieves state-of-the-art performance across a range of noise levels on both class-conditional image generation and visuomotor policy generation tasks.The code can be accessible via the project page https://robustdiffusionpolicy.github.io

Related papers

• Condition Errors Refinement in Autoregressive Image Generation with Diffusion Loss [56.120591983649824]
  We present a theoretical analysis of diffusion and autoregressive models with diffusion loss.<n>We show that patch denoising optimization in autoregressive models effectively mitigates condition errors and leads to a stable condition distribution.<n>We introduce a novel condition refinement approach based on Optimal Transport (OT) theory to address condition inconsistency''
  arXiv  Detail & Related papers  (2026-02-02T07:48:04Z)
• Mitigating the Noise Shift for Denoising Generative Models via Noise Awareness Guidance [54.88271057438763]
  Noise Awareness Guidance (NAG) is a correction method that explicitly steers sampling trajectories to remain consistent with the pre-defined noise schedule.<n>NAG consistently mitigates noise shift and substantially improves the generation quality of mainstream diffusion models.
  arXiv  Detail & Related papers  (2025-10-14T13:31:34Z)
• Unconditional Priors Matter! Improving Conditional Generation of Fine-Tuned Diffusion Models [10.542645300983878]
  We show that replacing unconditional noise in CFG with that predicted by the base model can significantly improve conditional generation.<n>We experimentally verify our claim with a range CFG-based conditional models for both image and video generation.
  arXiv  Detail & Related papers  (2025-03-26T05:11:38Z)
• Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis [62.07413805483241]
  Steered Diffusion is a framework for zero-shot conditional image generation using a diffusion model trained for unconditional generation. We present experiments using steered diffusion on several tasks including inpainting, colorization, text-guided semantic editing, and image super-resolution.
  arXiv  Detail & Related papers  (2023-09-30T02:03:22Z)
• Gradpaint: Gradient-Guided Inpainting with Diffusion Models [71.47496445507862]
  Denoising Diffusion Probabilistic Models (DDPMs) have recently achieved remarkable results in conditional and unconditional image generation. We present GradPaint, which steers the generation towards a globally coherent image. We generalizes well to diffusion models trained on various datasets, improving upon current state-of-the-art supervised and unsupervised methods.
  arXiv  Detail & Related papers  (2023-09-18T09:36:24Z)
• Instructed Diffuser with Temporal Condition Guidance for Offline Reinforcement Learning [71.24316734338501]
  We propose an effective temporally-conditional diffusion model coined Temporally-Composable diffuser (TCD) TCD extracts temporal information from interaction sequences and explicitly guides generation with temporal conditions. Our method reaches or matches the best performance compared with prior SOTA baselines.
  arXiv  Detail & Related papers  (2023-06-08T02:12:26Z)
• Conditional Generation from Unconditional Diffusion Models using Denoiser Representations [94.04631421741986]
  We propose adapting pre-trained unconditional diffusion models to new conditions using the learned internal representations of the denoiser network. We show that augmenting the Tiny ImageNet training set with synthetic images generated by our approach improves the classification accuracy of ResNet baselines by up to 8%.
  arXiv  Detail & Related papers  (2023-06-02T20:09:57Z)
• ShiftDDPMs: Exploring Conditional Diffusion Models by Shifting Diffusion Trajectories [144.03939123870416]
  We propose a novel conditional diffusion model by introducing conditions into the forward process. We use extra latent space to allocate an exclusive diffusion trajectory for each condition based on some shifting rules. We formulate our method, which we call textbfShiftDDPMs, and provide a unified point of view on existing related methods.
  arXiv  Detail & Related papers  (2023-02-05T12:48:21Z)
• On Conditioning the Input Noise for Controlled Image Generation with Diffusion Models [27.472482893004862]
  Conditional image generation has paved the way for several breakthroughs in image editing, generating stock photos and 3-D object generation. In this work, we explore techniques to condition diffusion models with carefully crafted input noise artifacts.
  arXiv  Detail & Related papers  (2022-05-08T13:18:14Z)
• Behavior of Keyword Spotting Networks Under Noisy Conditions [1.5425424751424208]
  Keywords spotting (KWS) is becoming a ubiquitous need with the advancement in artificial intelligence and smart devices. Recent work in this field have focused on several different architectures to achieve good results on datasets with low to moderate noise. We present an extensive comparison between state-of-the-art KWS networks under various noisy conditions.
  arXiv  Detail & Related papers  (2021-09-15T10:02:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.