Diffusion Models are Evolutionary Algorithms

• URL: http://arxiv.org/abs/2410.02543v2
• Date: Fri, 4 Oct 2024 01:55:52 GMT
• Title: Diffusion Models are Evolutionary Algorithms
• Authors: Yanbo Zhang, Benedikt Hartl, Hananel Hazan, Michael Levin,
• Abstract summary: We show that diffusion models inherently perform evolutionary algorithms, naturally encompassing selection, mutation, and reproductive isolation. We propose the Diffusion Evolution method: an evolutionary algorithm utilizing iterative denoising. We also introduce Latent Space Diffusion Evolution, which finds solutions for evolutionary tasks in high-dimensional complex parameter space.
• Score: 1.8299322342860518
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In a convergence of machine learning and biology, we reveal that diffusion models are evolutionary algorithms. By considering evolution as a denoising process and reversed evolution as diffusion, we mathematically demonstrate that diffusion models inherently perform evolutionary algorithms, naturally encompassing selection, mutation, and reproductive isolation. Building on this equivalence, we propose the Diffusion Evolution method: an evolutionary algorithm utilizing iterative denoising -- as originally introduced in the context of diffusion models -- to heuristically refine solutions in parameter spaces. Unlike traditional approaches, Diffusion Evolution efficiently identifies multiple optimal solutions and outperforms prominent mainstream evolutionary algorithms. Furthermore, leveraging advanced concepts from diffusion models, namely latent space diffusion and accelerated sampling, we introduce Latent Space Diffusion Evolution, which finds solutions for evolutionary tasks in high-dimensional complex parameter space while significantly reducing computational steps. This parallel between diffusion and evolution not only bridges two different fields but also opens new avenues for mutual enhancement, raising questions about open-ended evolution and potentially utilizing non-Gaussian or discrete diffusion models in the context of Diffusion Evolution.

Related papers

• G2D2: Gradient-guided Discrete Diffusion for image inverse problem solving [55.185588994883226]
  This paper presents a novel method for addressing linear inverse problems by leveraging image-generation models based on discrete diffusion as priors. To the best of our knowledge, this is the first approach to use discrete diffusion model-based priors for solving image inverse problems.
  arXiv  Detail & Related papers  (2024-10-09T06:18:25Z)
• The Tree of Diffusion Life: Evolutionary Embeddings to Understand the Generation Process of Diffusion Models [2.353466020397348]
  Tree of Diffusion Life (TDL) is a method to understand data evolution in the generative process of diffusion models. TDL samples a diffusion model's generative space via instances with varying iterations and employs image encoders to extract semantic meaning from these samples. It employs a novel evolutionary embedding algorithm that explicitly encodes the iterations while preserving the high-dimensional relations.
  arXiv  Detail & Related papers  (2024-06-25T11:05:26Z)
• An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization [59.63880337156392]
  Diffusion models have achieved tremendous success in computer vision, audio, reinforcement learning, and computational biology. Despite the significant empirical success, theory of diffusion models is very limited. This paper provides a well-rounded theoretical exposure for stimulating forward-looking theories and methods of diffusion models.
  arXiv  Detail & Related papers  (2024-04-11T14:07:25Z)
• EmoDM: A Diffusion Model for Evolutionary Multi-objective Optimization [22.374325061635112]
  This work proposes for the first time a diffusion model that can learn to perform evolutionary multi-objective search, called EmoDM. EmoDM can generate a set of non-dominated solutions for a new MOP by means of its reverse diffusion without further evolutionary search. Experimental results demonstrate the competitiveness of EmoDM in terms of both the search performance and computational efficiency.
  arXiv  Detail & Related papers  (2024-01-29T07:41:44Z)
• Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors [11.01598521921903]
  DDVI is a black-box variational inference algorithm for latent variable models. We introduce an expressive class of diffusion-based variational posteriors. We train these posteriors with a novel regularized evidence lower bound.
  arXiv  Detail & Related papers  (2024-01-05T10:27:44Z)
• A Variational Perspective on Solving Inverse Problems with Diffusion Models [101.831766524264]
  Inverse tasks can be formulated as inferring a posterior distribution over data. This is however challenging in diffusion models since the nonlinear and iterative nature of the diffusion process renders the posterior intractable. We propose a variational approach that by design seeks to approximate the true posterior distribution.
  arXiv  Detail & Related papers  (2023-05-07T23:00:47Z)
• Diffusion Models in Vision: A Survey [80.82832715884597]
  A diffusion model is a deep generative model that is based on two stages, a forward diffusion stage and a reverse diffusion stage. Diffusion models are widely appreciated for the quality and diversity of the generated samples, despite their known computational burdens.
  arXiv  Detail & Related papers  (2022-09-10T22:00:30Z)
• A Survey on Generative Diffusion Model [75.93774014861978]
  Diffusion models are an emerging class of deep generative models. They have certain limitations, including a time-consuming iterative generation process and confinement to high-dimensional Euclidean space. This survey presents a plethora of advanced techniques aimed at enhancing diffusion models.
  arXiv  Detail & Related papers  (2022-09-06T16:56:21Z)
• EvoVGM: A Deep Variational Generative Model for Evolutionary Parameter Estimation [0.0]
  We propose a method for a deep variational Bayesian generative model that jointly approximates the true posterior of local biological evolutionary parameters. We show the consistency and effectiveness of the method on synthetic sequence alignments simulated with several evolutionary scenarios and on a real virus sequence alignment.
  arXiv  Detail & Related papers  (2022-05-25T20:08:10Z)
• GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation [102.85440102147267]
  We propose a novel generative model named GeoDiff for molecular conformation prediction. We show that GeoDiff is superior or comparable to existing state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-03-06T09:47:01Z)
• Evolving Evolutionary Algorithms with Patterns [0.0]
  The model is based on the Multi Expression Programming (MEP) technique. Several evolutionary algorithms for function optimization are evolved by using the considered model.
  arXiv  Detail & Related papers  (2021-10-10T16:26:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.