Rapid optimization in high dimensional space by deep kernel learning augmented genetic algorithms

• URL: http://arxiv.org/abs/2410.03173v1
• Date: Fri, 4 Oct 2024 06:18:17 GMT
• Title: Rapid optimization in high dimensional space by deep kernel learning augmented genetic algorithms
• Authors: Mani Valleti, Aditya Raghavan, Sergei V. Kalinin,
• Abstract summary: Deep Kernel Learning (DKL) efficiently navigates the spaces of preselected candidate structures but lacks generative capabilities. This study introduces an approach that amalgamates the generative power of GAs to create new candidates with the efficiency of DKL-based surrogate models. We demonstrate the effectiveness of this approach through the optimization of the FerroSIM model, showcasing its broad applicability to diverse challenges.
• Score: 0.26716003713321473
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Exploration of complex high-dimensional spaces presents significant challenges in fields such as molecular discovery, process optimization, and supply chain management. Genetic Algorithms (GAs), while offering significant power for creating new candidate spaces, often entail high computational demands due to the need for evaluation of each new proposed solution. On the other hand, Deep Kernel Learning (DKL) efficiently navigates the spaces of preselected candidate structures but lacks generative capabilities. This study introduces an approach that amalgamates the generative power of GAs to create new candidates with the efficiency of DKL-based surrogate models to rapidly ascertain the behavior of new candidate spaces. This DKL-GA framework can be further used to build Bayesian Optimization (BO) workflows. We demonstrate the effectiveness of this approach through the optimization of the FerroSIM model, showcasing its broad applicability to diverse challenges, including molecular discovery and battery charging optimization.

Related papers

• Scalable Bayesian Optimization via Focalized Sparse Gaussian Processes [8.40647440727154]
  We argue that Bayesian optimization algorithms with sparse GPs can more efficiently allocate their representational power to relevant regions of the search space. We show that FocalBO can efficiently leverage large amounts of offline and online data to achieve state-of-the-art performance on robot morphology design and to control a 585-dimensional musculoskeletal system.
  arXiv  Detail & Related papers  (2024-12-29T06:36:15Z)
• GDSG: Graph Diffusion-based Solution Generator for Optimization Problems in MEC Networks [109.17835015018532]
  We present a Graph Diffusion-based Solution Generation (GDSG) method. This approach is designed to work with suboptimal datasets while converging to the optimal solution large probably. We build GDSG as a multi-task diffusion model utilizing a Graph Neural Network (GNN) to acquire the distribution of high-quality solutions.
  arXiv  Detail & Related papers  (2024-12-11T11:13:43Z)
• Faster Optimal Coalition Structure Generation via Offline Coalition Selection and Graph-Based Search [61.08720171136229]
  We present a novel algorithm, SMART, for the problem based on a hybridization of three innovative techniques. Two of these techniques are based on dynamic programming, where we show a powerful connection between the coalitions selected for evaluation and the performance of the algorithms. Our techniques bring a new way of approaching the problem and a new level of precision to the field.
  arXiv  Detail & Related papers  (2024-07-22T23:24:03Z)
• Localized Zeroth-Order Prompt Optimization [54.964765668688806]
  We propose a novel algorithm, namely localized zeroth-order prompt optimization (ZOPO) ZOPO incorporates a Neural Tangent Kernel-based derived Gaussian process into standard zeroth-order optimization for an efficient search of well-performing local optima in prompt optimization. Remarkably, ZOPO outperforms existing baselines in terms of both the optimization performance and the query efficiency.
  arXiv  Detail & Related papers  (2024-03-05T14:18:15Z)
• RTDK-BO: High Dimensional Bayesian Optimization with Reinforced Transformer Deep kernels [39.53062980223013]
  We combine recent developments in Deep Kernel Learning (DKL) and attention-based Transformer models to improve the modeling powers of GP surrogates with meta-learning. We propose a novel method for improving meta-learning BO surrogates by incorporating attention mechanisms into DKL. We combine this Transformer Deep Kernel with a learned acquisition function trained with continuous Soft Actor-Critic Reinforcement Learning to aid in exploration.
  arXiv  Detail & Related papers  (2023-10-05T21:37:20Z)
• A new derivative-free optimization method: Gaussian Crunching Search [0.0]
  We introduce a novel optimization method called Gaussian Crunching Search (GCS) Inspired by the behaviour of particles in a Gaussian distribution, GCS aims to efficiently explore the solution space and converge towards the global optimum. This research paper serves as a valuable resource for researchers, practitioners, and students interested in optimization.
  arXiv  Detail & Related papers  (2023-07-24T16:17:53Z)
• An Empirical Evaluation of Zeroth-Order Optimization Methods on AI-driven Molecule Optimization [78.36413169647408]
  We study the effectiveness of various ZO optimization methods for optimizing molecular objectives. We show the advantages of ZO sign-based gradient descent (ZO-signGD) We demonstrate the potential effectiveness of ZO optimization methods on widely used benchmark tasks from the Guacamol suite.
  arXiv  Detail & Related papers  (2022-10-27T01:58:10Z)
• Designing Biological Sequences via Meta-Reinforcement Learning and Bayesian Optimization [68.28697120944116]
  We train an autoregressive generative model via Meta-Reinforcement Learning to propose promising sequences for selection. We pose this problem as that of finding an optimal policy over a distribution of MDPs induced by sampling subsets of the data. Our in-silico experiments show that meta-learning over such ensembles provides robustness against reward misspecification and achieves competitive results.
  arXiv  Detail & Related papers  (2022-09-13T18:37:27Z)
• Surrogate-Based Black-Box Optimization Method for Costly Molecular Properties [0.0]
  We propose a surrogate-based black box optimization method to tackle jointly the optimization and machine learning problems. We show that our approach can successfully optimize a costly property of interest much faster than a purely metaheuristic approach.
  arXiv  Detail & Related papers  (2021-10-01T15:28:15Z)
• JANUS: Parallel Tempered Genetic Algorithm Guided by Deep Neural Networks for Inverse Molecular Design [1.6114012813668934]
  Inverse molecular design, i.e., designing molecules with specific target properties, can be posed as an optimization problem. Janus is a genetic algorithm inspired by parallel tempering that propagates two populations, one for exploration and another for exploitation. Janus is augmented by a deep neural network that approximates molecular properties via active learning for enhanced sampling of the chemical space.
  arXiv  Detail & Related papers  (2021-06-07T23:41:34Z)
• AdaLead: A simple and robust adaptive greedy search algorithm for sequence design [55.41644538483948]
  We develop an easy-to-directed, scalable, and robust evolutionary greedy algorithm (AdaLead) AdaLead is a remarkably strong benchmark that out-competes more complex state of the art approaches in a variety of biologically motivated sequence design challenges.
  arXiv  Detail & Related papers  (2020-10-05T16:40:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.