Mean-field Chaos Diffusion Models

• URL: http://arxiv.org/abs/2406.05396v1
• Date: Sat, 8 Jun 2024 08:24:06 GMT
• Title: Mean-field Chaos Diffusion Models
• Authors: Sungwoo Park, Dongjun Kim, Ahmed Alaa,
• Abstract summary: We introduce a new class of score-based generative models (SGMs) designed to handle high-cardinality data distributions. We present mean-field chaos diffusion models (MF-CDMs), which address the curse of dimensionality inherent in high-cardinality data.
• Score: 19.289421150924206
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we introduce a new class of score-based generative models (SGMs) designed to handle high-cardinality data distributions by leveraging concepts from mean-field theory. We present mean-field chaos diffusion models (MF-CDMs), which address the curse of dimensionality inherent in high-cardinality data by utilizing the propagation of chaos property of interacting particles. By treating high-cardinality data as a large stochastic system of interacting particles, we develop a novel score-matching method for infinite-dimensional chaotic particle systems and propose an approximation scheme that employs a subdivision strategy for efficient training. Our theoretical and empirical results demonstrate the scalability and effectiveness of MF-CDMs for managing large high-cardinality data structures, such as 3D point clouds.

Related papers

• Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction [88.65168366064061]
  We introduce Discrete Denoising Posterior Prediction (DDPP), a novel framework that casts the task of steering pre-trained MDMs as a problem of probabilistic inference. Our framework leads to a family of three novel objectives that are all simulation-free, and thus scalable. We substantiate our designs via wet-lab validation, where we observe transient expression of reward-optimized protein sequences.
  arXiv  Detail & Related papers  (2024-10-10T17:18:30Z)
• Neural Message Passing Induced by Energy-Constrained Diffusion [79.9193447649011]
  We propose an energy-constrained diffusion model as a principled interpretable framework for understanding the mechanism of MPNNs. We show that the new model can yield promising performance for cases where the data structures are observed (as a graph), partially observed or completely unobserved.
  arXiv  Detail & Related papers  (2024-09-13T17:54:41Z)
• Shedding Light on Large Generative Networks: Estimating Epistemic Uncertainty in Diffusion Models [15.352556466952477]
  Generative diffusion models are notable for their large parameter count (exceeding 100 million) and operation within high-dimensional image spaces. We introduce an innovative framework, Diffusion Ensembles for Capturing Uncertainty (DECU), designed for estimating epistemic uncertainty for diffusion models.
  arXiv  Detail & Related papers  (2024-06-05T14:03:21Z)
• Diffusion-C: Unveiling the Generative Challenges of Diffusion Models through Corrupted Data [2.7624021966289605]
  "Diffusion-C" is a foundational methodology to analyze the generative restrictions of Diffusion Models. Within the milieu of generative models under the Diffusion taxonomy, DDPM emerges as a paragon, consistently exhibiting superior performance metrics. The vulnerability of Diffusion Models to these particular corruptions is significantly influenced by topological and statistical similarities.
  arXiv  Detail & Related papers  (2023-12-14T12:01:51Z)
• CamoDiffusion: Camouflaged Object Detection via Conditional Diffusion Models [72.93652777646233]
  Camouflaged Object Detection (COD) is a challenging task in computer vision due to the high similarity between camouflaged objects and their surroundings. We propose a new paradigm that treats COD as a conditional mask-generation task leveraging diffusion models. Our method, dubbed CamoDiffusion, employs the denoising process of diffusion models to iteratively reduce the noise of the mask.
  arXiv  Detail & Related papers  (2023-05-29T07:49:44Z)
• Hierarchical Integration Diffusion Model for Realistic Image Deblurring [71.76410266003917]
  Diffusion models (DMs) have been introduced in image deblurring and exhibited promising performance. We propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring. Experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-22T12:18:20Z)
• SPIRiT-Diffusion: Self-Consistency Driven Diffusion Model for Accelerated MRI [14.545736786515837]
  We introduce SPIRiT-Diffusion, a diffusion model for k-space inspired by the iterative self-consistent SPIRiT method. We evaluate the proposed SPIRiT-Diffusion method using a 3D joint intracranial and carotid vessel wall imaging dataset.
  arXiv  Detail & Related papers  (2023-04-11T08:43:52Z)
• Information-Theoretic Diffusion [18.356162596599436]
  Denoising diffusion models have spurred significant gains in density modeling and image generation. We introduce a new mathematical foundation for diffusion models inspired by classic results in information theory.
  arXiv  Detail & Related papers  (2023-02-07T23:03:07Z)
• Modiff: Action-Conditioned 3D Motion Generation with Denoising Diffusion Probabilistic Models [58.357180353368896]
  We propose a conditional paradigm that benefits from the denoising diffusion probabilistic model (DDPM) to tackle the problem of realistic and diverse action-conditioned 3D skeleton-based motion generation. We are a pioneering attempt that uses DDPM to synthesize a variable number of motion sequences conditioned on a categorical action.
  arXiv  Detail & Related papers  (2023-01-10T13:15:42Z)
• Training Deep Energy-Based Models with f-Divergence Minimization [113.97274898282343]
  Deep energy-based models (EBMs) are very flexible in distribution parametrization but computationally challenging. We propose a general variational framework termed f-EBM to train EBMs using any desired f-divergence. Experimental results demonstrate the superiority of f-EBM over contrastive divergence, as well as the benefits of training EBMs using f-divergences other than KL.
  arXiv  Detail & Related papers  (2020-03-06T23:11:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.