Heuristically Adaptive Diffusion-Model Evolutionary Strategy

• URL: http://arxiv.org/abs/2411.13420v1
• Date: Wed, 20 Nov 2024 16:06:28 GMT
• Title: Heuristically Adaptive Diffusion-Model Evolutionary Strategy
• Authors: Benedikt Hartl, Yanbo Zhang, Hananel Hazan, Michael Levin,
• Abstract summary: Diffusion Models represent a significant advancement in generative modeling. Our research reveals a fundamental connection between diffusion models and evolutionary algorithms. Our framework marks a major algorithmic transition, offering increased flexibility, precision, and control in evolutionary optimization processes.
• Score: 1.8299322342860518
• License:
• Abstract: Diffusion Models represent a significant advancement in generative modeling, employing a dual-phase process that first degrades domain-specific information via Gaussian noise and restores it through a trainable model. This framework enables pure noise-to-data generation and modular reconstruction of, images or videos. Concurrently, evolutionary algorithms employ optimization methods inspired by biological principles to refine sets of numerical parameters encoding potential solutions to rugged objective functions. Our research reveals a fundamental connection between diffusion models and evolutionary algorithms through their shared underlying generative mechanisms: both methods generate high-quality samples via iterative refinement on random initial distributions. By employing deep learning-based diffusion models as generative models across diverse evolutionary tasks and iteratively refining diffusion models with heuristically acquired databases, we can iteratively sample potentially better-adapted offspring parameters, integrating them into successive generations of the diffusion model. This approach achieves efficient convergence toward high-fitness parameters while maintaining explorative diversity. Diffusion models introduce enhanced memory capabilities into evolutionary algorithms, retaining historical information across generations and leveraging subtle data correlations to generate refined samples. We elevate evolutionary algorithms from procedures with shallow heuristics to frameworks with deep memory. By deploying classifier-free guidance for conditional sampling at the parameter level, we achieve precise control over evolutionary search dynamics to further specific genotypical, phenotypical, or population-wide traits. Our framework marks a major heuristic and algorithmic transition, offering increased flexibility, precision, and control in evolutionary optimization processes.

Related papers

• Energy-Based Diffusion Language Models for Text Generation [126.23425882687195]
  Energy-based Diffusion Language Model (EDLM) is an energy-based model operating at the full sequence level for each diffusion step. Our framework offers a 1.3$times$ sampling speedup over existing diffusion models.
  arXiv  Detail & Related papers  (2024-10-28T17:25:56Z)
• Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design [56.957070405026194]
  We propose an algorithm that enables direct backpropagation of rewards through entire trajectories generated by diffusion models. DRAKES can generate sequences that are both natural-like and yield high rewards.
  arXiv  Detail & Related papers  (2024-10-17T15:10:13Z)
• Controlling the Fidelity and Diversity of Deep Generative Models via Pseudo Density [70.14884528360199]
  We introduce an approach to bias deep generative models, such as GANs and diffusion models, towards generating data with enhanced fidelity or increased diversity. Our approach involves manipulating the distribution of training and generated data through a novel metric for individual samples, named pseudo density.
  arXiv  Detail & Related papers  (2024-07-11T16:46:04Z)
• Improving Out-of-Distribution Robustness of Classifiers via Generative Interpolation [56.620403243640396]
  Deep neural networks achieve superior performance for learning from independent and identically distributed (i.i.d.) data. However, their performance deteriorates significantly when handling out-of-distribution (OoD) data. We develop a simple yet effective method called Generative Interpolation to fuse generative models trained from multiple domains for synthesizing diverse OoD samples.
  arXiv  Detail & Related papers  (2023-07-23T03:53:53Z)
• Protein Design with Guided Discrete Diffusion [67.06148688398677]
  A popular approach to protein design is to combine a generative model with a discriminative model for conditional sampling. We propose diffusioN Optimized Sampling (NOS), a guidance method for discrete diffusion models. NOS makes it possible to perform design directly in sequence space, circumventing significant limitations of structure-based methods.
  arXiv  Detail & Related papers  (2023-05-31T16:31:24Z)
• Diffusion Probabilistic Model Based Accurate and High-Degree-of-Freedom Metasurface Inverse Design [14.18549701990854]
  Inverse design methods based on optimization algorithms have been introduced to design metamaterials. Deep learning methods represented by Generative Adversarial Networks (GANs) have been applied to inverse design of metamaterials. This paper proposes a novel metamaterial inverse design method based on the diffusion probability theory.
  arXiv  Detail & Related papers  (2023-04-25T08:25:23Z)
• A Reparameterized Discrete Diffusion Model for Text Generation [39.0145272152805]
  This work studies discrete diffusion probabilistic models with applications to natural language generation. We derive an alternative yet equivalent formulation of the sampling from discrete diffusion processes. We conduct extensive experiments to evaluate the text generation capability of our model, demonstrating significant improvements over existing diffusion models.
  arXiv  Detail & Related papers  (2023-02-11T16:26:57Z)
• A Survey on Generative Diffusion Model [75.93774014861978]
  Diffusion models are an emerging class of deep generative models. They have certain limitations, including a time-consuming iterative generation process and confinement to high-dimensional Euclidean space. This survey presents a plethora of advanced techniques aimed at enhancing diffusion models.
  arXiv  Detail & Related papers  (2022-09-06T16:56:21Z)
• EvoVGM: A Deep Variational Generative Model for Evolutionary Parameter Estimation [0.0]
  We propose a method for a deep variational Bayesian generative model that jointly approximates the true posterior of local biological evolutionary parameters. We show the consistency and effectiveness of the method on synthetic sequence alignments simulated with several evolutionary scenarios and on a real virus sequence alignment.
  arXiv  Detail & Related papers  (2022-05-25T20:08:10Z)
• Deep Evolutionary Learning for Molecular Design [1.8047694351309207]
  We propose a deep evolutionary learning process that integrates fragment-based deep generative model and multi-objective evolutionary computation for molecular design. Our approach enables (1) evolutionary operations in the latent space of the generative model, rather than the structural space, to generate novel promising molecular structures for the next evolutionary generation.
  arXiv  Detail & Related papers  (2020-12-28T03:15:46Z)
• Evolutionary Variational Optimization of Generative Models [0.0]
  We combine two popular optimization approaches to derive learning algorithms for generative models: variational optimization and evolutionary algorithms. We show that evolutionary algorithms can effectively and efficiently optimize the variational bound. In the category of "zero-shot" learning, we observed the evolutionary variational algorithm to significantly improve the state-of-the-art in many benchmark settings.
  arXiv  Detail & Related papers  (2020-12-22T19:06:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.