A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data

• URL: http://arxiv.org/abs/2402.16991v3
• Date: Tue, 24 Dec 2024 02:17:39 GMT
• Title: A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data
• Authors: Antonio Sclocchi, Alessandro Favero, Matthieu Wyart,
• Abstract summary: Recent advancements show that diffusion models can generate high-quality images. We study this phenomenon in a hierarchical generative model of data. We find that the backward diffusion process acting after a time $t$ is governed by a phase transition.
• Score: 51.03144354630136
• License:
• Abstract: Understanding the structure of real data is paramount in advancing modern deep-learning methodologies. Natural data such as images are believed to be composed of features organized in a hierarchical and combinatorial manner, which neural networks capture during learning. Recent advancements show that diffusion models can generate high-quality images, hinting at their ability to capture this underlying compositional structure. We study this phenomenon in a hierarchical generative model of data. We find that the backward diffusion process acting after a time $t$ is governed by a phase transition at some threshold time, where the probability of reconstructing high-level features, like the class of an image, suddenly drops. Instead, the reconstruction of low-level features, such as specific details of an image, evolves smoothly across the whole diffusion process. This result implies that at times beyond the transition, the class has changed, but the generated sample may still be composed of low-level elements of the initial image. We validate these theoretical insights through numerical experiments on class-unconditional ImageNet diffusion models. Our analysis characterizes the relationship between time and scale in diffusion models and puts forward generative models as powerful tools to model combinatorial data properties.

Related papers

• How compositional generalization and creativity improve as diffusion models are trained [82.08869888944324]
  How many samples do generative models need to learn the composition rules, so as to produce a number of novel data? We consider diffusion models trained on simple context-free grammars - tree-like graphical models used to represent the structure of data such as language and images. We demonstrate that diffusion models learn compositional rules with the sample complexity required for clustering features with statistically similar context, a process similar to the word2vec.
  arXiv  Detail & Related papers  (2025-02-17T18:06:33Z)
• Nested Diffusion Models Using Hierarchical Latent Priors [23.605302440082994]
  We introduce nested diffusion models, an efficient and powerful hierarchical generative framework. Our approach employs a series of diffusion models to progressively generate latent variables at different semantic levels. To construct these latent variables, we leverage a pre-trained visual encoder, which learns strong semantic visual representations.
  arXiv  Detail & Related papers  (2024-12-08T16:13:39Z)
• Probing the Latent Hierarchical Structure of Data via Diffusion Models [47.56642214162824]
  We show that experiments in diffusion-based models are a promising tool to probe the latent structure of data. We confirm this prediction in both text and image datasets using state-of-the-art diffusion models. Our results show how latent variable changes manifest in the data and establish how to measure these effects in real data.
  arXiv  Detail & Related papers  (2024-10-17T17:08:39Z)
• How Diffusion Models Learn to Factorize and Compose [14.161975556325796]
  Diffusion models are capable of generating photo-realistic images that combine elements which likely do not appear together in the training set. We investigate whether and when diffusion models learn semantically meaningful and factorized representations of composable features.
  arXiv  Detail & Related papers  (2024-08-23T17:59:03Z)
• Training Class-Imbalanced Diffusion Model Via Overlap Optimization [55.96820607533968]
  Diffusion models trained on real-world datasets often yield inferior fidelity for tail classes. Deep generative models, including diffusion models, are biased towards classes with abundant training images. We propose a method based on contrastive learning to minimize the overlap between distributions of synthetic images for different classes.
  arXiv  Detail & Related papers  (2024-02-16T16:47:21Z)
• Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis [62.07413805483241]
  Steered Diffusion is a framework for zero-shot conditional image generation using a diffusion model trained for unconditional generation. We present experiments using steered diffusion on several tasks including inpainting, colorization, text-guided semantic editing, and image super-resolution.
  arXiv  Detail & Related papers  (2023-09-30T02:03:22Z)
• ChiroDiff: Modelling chirographic data with Diffusion Models [132.5223191478268]
  We introduce a powerful model-class namely "Denoising Diffusion Probabilistic Models" or DDPMs for chirographic data. Our model named "ChiroDiff", being non-autoregressive, learns to capture holistic concepts and therefore remains resilient to higher temporal sampling rate.
  arXiv  Detail & Related papers  (2023-04-07T15:17:48Z)
• Compositional Visual Generation with Composable Diffusion Models [80.75258849913574]
  We propose an alternative structured approach for compositional generation using diffusion models. An image is generated by composing a set of diffusion models, with each of them modeling a certain component of the image. The proposed method can generate scenes at test time that are substantially more complex than those seen in training.
  arXiv  Detail & Related papers  (2022-06-03T17:47:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.