Ambient Diffusion Omni: Training Good Models with Bad Data

• URL: http://arxiv.org/abs/2506.10038v1
• Date: Tue, 10 Jun 2025 22:37:39 GMT
• Title: Ambient Diffusion Omni: Training Good Models with Bad Data
• Authors: Giannis Daras, Adrian Rodriguez-Munoz, Adam Klivans, Antonio Torralba, Constantinos Daskalakis,
• Abstract summary: We show how to use low-quality, synthetic, and out-of-distribution images to improve the quality of a diffusion model.<n>We present Ambient Omni, a principled framework to train diffusion models that can extract signal from all available images.
• Score: 45.821861121026394
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We show how to use low-quality, synthetic, and out-of-distribution images to improve the quality of a diffusion model. Typically, diffusion models are trained on curated datasets that emerge from highly filtered data pools from the Web and other sources. We show that there is immense value in the lower-quality images that are often discarded. We present Ambient Diffusion Omni, a simple, principled framework to train diffusion models that can extract signal from all available images during training. Our framework exploits two properties of natural images -- spectral power law decay and locality. We first validate our framework by successfully training diffusion models with images synthetically corrupted by Gaussian blur, JPEG compression, and motion blur. We then use our framework to achieve state-of-the-art ImageNet FID, and we show significant improvements in both image quality and diversity for text-to-image generative modeling. The core insight is that noise dampens the initial skew between the desired high-quality distribution and the mixed distribution we actually observe. We provide rigorous theoretical justification for our approach by analyzing the trade-off between learning from biased data versus limited unbiased data across diffusion times.

Related papers

• Integrating Amortized Inference with Diffusion Models for Learning Clean Distribution from Corrupted Images [19.957503854446735]
  Diffusion models (DMs) have emerged as powerful generative models for solving inverse problems. FlowDiff is a joint training paradigm that leverages a conditional normalizing flow model to facilitate the training of diffusion models on corrupted data sources. Our experiment shows that FlowDiff can effectively learn clean distributions across a wide range of corrupted data sources.
  arXiv  Detail & Related papers  (2024-07-15T18:33:20Z)
• Training Class-Imbalanced Diffusion Model Via Overlap Optimization [55.96820607533968]
  Diffusion models trained on real-world datasets often yield inferior fidelity for tail classes. Deep generative models, including diffusion models, are biased towards classes with abundant training images. We propose a method based on contrastive learning to minimize the overlap between distributions of synthetic images for different classes.
  arXiv  Detail & Related papers  (2024-02-16T16:47:21Z)
• Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis [62.07413805483241]
  Steered Diffusion is a framework for zero-shot conditional image generation using a diffusion model trained for unconditional generation. We present experiments using steered diffusion on several tasks including inpainting, colorization, text-guided semantic editing, and image super-resolution.
  arXiv  Detail & Related papers  (2023-09-30T02:03:22Z)
• Unbiased Image Synthesis via Manifold Guidance in Diffusion Models [9.531220208352252]
  Diffusion Models often inadvertently favor certain data attributes, undermining the diversity of generated images. We propose a plug-and-play method named Manifold Sampling Guidance, which is also the first unsupervised method to mitigate bias issue in DDPMs.
  arXiv  Detail & Related papers  (2023-07-17T02:03:17Z)
• Generating images of rare concepts using pre-trained diffusion models [32.5337654536764]
  Text-to-image diffusion models can synthesize high-quality images, but they have various limitations. We show that their limitation is partly due to the long-tail nature of their training data. We show that rare concepts can be correctly generated by carefully selecting suitable generation seeds in the noise space.
  arXiv  Detail & Related papers  (2023-04-27T20:55:38Z)
• Your Diffusion Model is Secretly a Zero-Shot Classifier [90.40799216880342]
  We show that density estimates from large-scale text-to-image diffusion models can be leveraged to perform zero-shot classification. Our generative approach to classification attains strong results on a variety of benchmarks. Our results are a step toward using generative over discriminative models for downstream tasks.
  arXiv  Detail & Related papers  (2023-03-28T17:59:56Z)
• Extracting Training Data from Diffusion Models [77.11719063152027]
  We show that diffusion models memorize individual images from their training data and emit them at generation time. With a generate-and-filter pipeline, we extract over a thousand training examples from state-of-the-art models. We train hundreds of diffusion models in various settings to analyze how different modeling and data decisions affect privacy.
  arXiv  Detail & Related papers  (2023-01-30T18:53:09Z)
• SinDiffusion: Learning a Diffusion Model from a Single Natural Image [159.4285444680301]
  We present SinDiffusion, leveraging denoising diffusion models to capture internal distribution of patches from a single natural image. It is based on two core designs. First, SinDiffusion is trained with a single model at a single scale instead of multiple models with progressive growing of scales. Second, we identify that a patch-level receptive field of the diffusion network is crucial and effective for capturing the image's patch statistics.
  arXiv  Detail & Related papers  (2022-11-22T18:00:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.