Heat Death of Generative Models in Closed-Loop Learning

• URL: http://arxiv.org/abs/2404.02325v2
• Date: Wed, 28 Aug 2024 22:54:15 GMT
• Title: Heat Death of Generative Models in Closed-Loop Learning
• Authors: Matteo Marchi, Stefano Soatto, Pratik Chaudhari, Paulo Tabuada,
• Abstract summary: We study the learning dynamics of generative models that are fed back their own produced content in addition to their original training dataset. We show that, unless a sufficient amount of external data is introduced at each iteration, any non-trivial temperature leads the model to degenerate.
• Score: 63.83608300361159
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Improvement and adoption of generative machine learning models is rapidly accelerating, as exemplified by the popularity of LLMs (Large Language Models) for text, and diffusion models for image generation. As generative models become widespread, data they generate is incorporated into shared content through the public web. This opens the question of what happens when data generated by a model is fed back to the model in subsequent training campaigns. This is a question about the stability of the training process, whether the distribution of publicly accessible content, which we refer to as "knowledge", remains stable or collapses. Small scale empirical experiments reported in the literature show that this closed-loop training process is prone to degenerating. Models may start producing gibberish data, or sample from only a small subset of the desired data distribution (a phenomenon referred to as mode collapse). So far there has been only limited theoretical understanding of this process, in part due to the complexity of the deep networks underlying these generative models. The aim of this paper is to provide insights into this process (that we refer to as "generative closed-loop learning") by studying the learning dynamics of generative models that are fed back their own produced content in addition to their original training dataset. The sampling of many of these models can be controlled via a "temperature" parameter. Using dynamical systems tools, we show that, unless a sufficient amount of external data is introduced at each iteration, any non-trivial temperature leads the model to asymptotically degenerate. In fact, either the generative distribution collapses to a small set of outputs or becomes uniform over a large set of outputs.

Related papers

• Model Integrity when Unlearning with T2I Diffusion Models [11.321968363411145]
  We propose approximate Machine Unlearning algorithms to reduce the generation of specific types of images, characterized by samples from a forget distribution'' We then propose unlearning algorithms that demonstrate superior effectiveness in preserving model integrity compared to existing baselines.
  arXiv  Detail & Related papers  (2024-11-04T13:15:28Z)
• Ablation Based Counterfactuals [7.481286710933861]
  Ablation Based Counterfactuals (ABC) is a method of performing counterfactual analysis that relies on model ablation rather than model retraining. We demonstrate how we can construct a model like this using an ensemble of diffusion models. We then use this model to study the limits of training data attribution by enumerating full counterfactual landscapes.
  arXiv  Detail & Related papers  (2024-06-12T06:22:51Z)
• Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data [49.73114504515852]
  We show that replacing the original real data by each generation's synthetic data does indeed tend towards model collapse. We demonstrate that accumulating the successive generations of synthetic data alongside the original real data avoids model collapse.
  arXiv  Detail & Related papers  (2024-04-01T18:31:24Z)
• Towards Theoretical Understandings of Self-Consuming Generative Models [56.84592466204185]
  This paper tackles the emerging challenge of training generative models within a self-consuming loop. We construct a theoretical framework to rigorously evaluate how this training procedure impacts the data distributions learned by future models. We present results for kernel density estimation, delivering nuanced insights such as the impact of mixed data training on error propagation.
  arXiv  Detail & Related papers  (2024-02-19T02:08:09Z)
• Continual Learning of Diffusion Models with Generative Distillation [34.52513912701778]
  Diffusion models are powerful generative models that achieve state-of-the-art performance in image synthesis. In this paper, we propose generative distillation, an approach that distils the entire reverse process of a diffusion model.
  arXiv  Detail & Related papers  (2023-11-23T14:33:03Z)
• On the Stability of Iterative Retraining of Generative Models on their own Data [56.153542044045224]
  We study the impact of training generative models on mixed datasets. We first prove the stability of iterative training under the condition that the initial generative models approximate the data distribution well enough. We empirically validate our theory on both synthetic and natural images by iteratively training normalizing flows and state-of-the-art diffusion models.
  arXiv  Detail & Related papers  (2023-09-30T16:41:04Z)
• Learning to Jump: Thinning and Thickening Latent Counts for Generative Modeling [69.60713300418467]
  Learning to jump is a general recipe for generative modeling of various types of data. We demonstrate when learning to jump is expected to perform comparably to learning to denoise, and when it is expected to perform better.
  arXiv  Detail & Related papers  (2023-05-28T05:38:28Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.