Generating on Generated: An Approach Towards Self-Evolving Diffusion Models

• URL: http://arxiv.org/abs/2502.09963v1
• Date: Fri, 14 Feb 2025 07:41:47 GMT
• Title: Generating on Generated: An Approach Towards Self-Evolving Diffusion Models
• Authors: Xulu Zhang, Xiaoyong Wei, Jinlin Wu, Jiaxin Wu, Zhaoxiang Zhang, Zhen Lei, Qing Li,
• Abstract summary: Recursive Self-Improvement (RSI) enables intelligence systems to autonomously refine their capabilities. This paper explores the application of RSI in text-to-image diffusion models, addressing the challenge of training collapse caused by synthetic data.
• Score: 58.05857658085845
• License:
• Abstract: Recursive Self-Improvement (RSI) enables intelligence systems to autonomously refine their capabilities. This paper explores the application of RSI in text-to-image diffusion models, addressing the challenge of training collapse caused by synthetic data. We identify two key factors contributing to this collapse: the lack of perceptual alignment and the accumulation of generative hallucinations. To mitigate these issues, we propose three strategies: (1) a prompt construction and filtering pipeline designed to facilitate the generation of perceptual aligned data, (2) a preference sampling method to identify human-preferred samples and filter out generative hallucinations, and (3) a distribution-based weighting scheme to penalize selected samples with hallucinatory errors. Our extensive experiments validate the effectiveness of these approaches.

Related papers

• DiffATR: Diffusion-based Generative Modeling for Audio-Text Retrieval [49.076590578101985]
  We present a diffusion-based ATR framework (DiffATR) that generates joint distribution from noise. Experiments on the AudioCaps and Clotho datasets with superior performances, verify the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-09-16T06:33:26Z)
• Controlling the Fidelity and Diversity of Deep Generative Models via Pseudo Density [70.14884528360199]
  We introduce an approach to bias deep generative models, such as GANs and diffusion models, towards generating data with enhanced fidelity or increased diversity. Our approach involves manipulating the distribution of training and generated data through a novel metric for individual samples, named pseudo density.
  arXiv  Detail & Related papers  (2024-07-11T16:46:04Z)
• When in Doubt, Cascade: Towards Building Efficient and Capable Guardrails [19.80434777786657]
  We develop a synthetic pipeline to generate targeted and labeled data. We show that our method achieves competitive performance with a fraction of the cost in compute.
  arXiv  Detail & Related papers  (2024-07-08T18:39:06Z)
• DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception [78.26734070960886]
  Current perceptive models heavily depend on resource-intensive datasets. We introduce perception-aware loss (P.A. loss) through segmentation, improving both quality and controllability. Our method customizes data augmentation by extracting and utilizing perception-aware attribute (P.A. Attr) during generation.
  arXiv  Detail & Related papers  (2024-03-20T04:58:03Z)
• RealNet: A Feature Selection Network with Realistic Synthetic Anomaly for Anomaly Detection [7.626097310990373]
  We introduce RealNet, a feature reconstruction network with realistic synthetic anomaly and adaptive feature selection. We develop Anomaly-aware Features Selection (AFS) and Reconstruction Residuals Selection (RRS) Our results demonstrate significant improvements in both Image AUROC and Pixel AUROC compared to the current state-o-the-art methods.
  arXiv  Detail & Related papers  (2024-03-09T12:25:01Z)
• Fair Sampling in Diffusion Models through Switching Mechanism [5.560136885815622]
  We propose a fairness-aware sampling method called textitattribute switching mechanism for diffusion models. We mathematically prove and experimentally demonstrate the effectiveness of the proposed method on two key aspects.
  arXiv  Detail & Related papers  (2024-01-06T06:55:26Z)
• The Score-Difference Flow for Implicit Generative Modeling [1.1929584800629673]
  Implicit generative modeling aims to produce samples of synthetic data matching a target data distribution. Recent work has approached the IGM problem from the perspective of pushing synthetic source data toward the target distribution. We present the score difference between arbitrary target and source distributions as a flow that optimally reduces the Kullback-Leibler divergence between them.
  arXiv  Detail & Related papers  (2023-04-25T15:21:12Z)
• Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective [72.55093886515824]
  We introduce a causal formalism of motion forecasting, which casts the problem as a dynamic process with three groups of latent variables. We devise a modular architecture that factorizes the representations of invariant mechanisms and style confounders to approximate a causal graph. Experiment results on synthetic and real datasets show that our three proposed components significantly improve the robustness and reusability of the learned motion representations.
  arXiv  Detail & Related papers  (2021-11-29T18:59:09Z)
• Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection [54.38512834521367]
  We study contrast candidate generation and selection as a model-agnostic post-processing technique. We learn a discriminative correction model by generating alternative candidate summaries. This model is then used to select the best candidate as the final output summary.
  arXiv  Detail & Related papers  (2021-04-19T05:39:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.