ADO-LLM: Analog Design Bayesian Optimization with In-Context Learning of Large Language Models

• URL: http://arxiv.org/abs/2406.18770v1
• Date: Wed, 26 Jun 2024 21:42:50 GMT
• Title: ADO-LLM: Analog Design Bayesian Optimization with In-Context Learning of Large Language Models
• Authors: Yuxuan Yin, Yu Wang, Boxun Xu, Peng Li,
• Abstract summary: This paper presents ADO-LLM, the first work integrating large language models (LLMs) with Bayesian Optimization for analog design optimization. ADO-LLM leverages the LLM's ability to infuse domain knowledge to rapidly generate viable design points to remedy BO's inefficiency in finding high value design areas. We evaluate the proposed framework on two different types of analog circuits and demonstrate notable improvements in design efficiency and effectiveness.
• Score: 5.642568057913696
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Analog circuit design requires substantial human expertise and involvement, which is a significant roadblock to design productivity. Bayesian Optimization (BO), a popular machine learning based optimization strategy, has been leveraged to automate analog design given its applicability across various circuit topologies and technologies. Traditional BO methods employ black box Gaussian Process surrogate models and optimized labeled data queries to find optimization solutions by trading off between exploration and exploitation. However, the search for the optimal design solution in BO can be expensive from both a computational and data usage point of view, particularly for high dimensional optimization problems. This paper presents ADO-LLM, the first work integrating large language models (LLMs) with Bayesian Optimization for analog design optimization. ADO-LLM leverages the LLM's ability to infuse domain knowledge to rapidly generate viable design points to remedy BO's inefficiency in finding high value design areas specifically under the limited design space coverage of the BO's probabilistic surrogate model. In the meantime, sampling of design points evaluated in the iterative BO process provides quality demonstrations for the LLM to generate high quality design points while leveraging infused broad design knowledge. Furthermore, the diversity brought by BO's exploration enriches the contextual understanding of the LLM and allows it to more broadly search in the design space and prevent repetitive and redundant suggestions. We evaluate the proposed framework on two different types of analog circuits and demonstrate notable improvements in design efficiency and effectiveness.

Related papers

• Latent Energy-Based Odyssey: Black-Box Optimization via Expanded Exploration in the Energy-Based Latent Space [65.44449711359724]
  High-dimensional and highly-multimodal input design space of black-box function pose inherent challenges for existing methods. We consider finding a latent space that serves as a compressed yet accurate representation of the design-value joint space. We propose Noise-intensified Telescoping density-Ratio Estimation scheme for variational learning of an accurate latent space model.
  arXiv  Detail & Related papers  (2024-05-27T00:11:53Z)
• Diffusion Model for Data-Driven Black-Box Optimization [54.25693582870226]
  We focus on diffusion models, a powerful generative AI technology, and investigate their potential for black-box optimization. We study two practical types of labels: 1) noisy measurements of a real-valued reward function and 2) human preference based on pairwise comparisons. Our proposed method reformulates the design optimization problem into a conditional sampling problem, which allows us to leverage the power of diffusion models.
  arXiv  Detail & Related papers  (2024-03-20T00:41:12Z)
• Enhanced Bayesian Optimization via Preferential Modeling of Abstract Properties [49.351577714596544]
  We propose a human-AI collaborative Bayesian framework to incorporate expert preferences about unmeasured abstract properties into surrogate modeling. We provide an efficient strategy that can also handle any incorrect/misleading expert bias in preferential judgments.
  arXiv  Detail & Related papers  (2024-02-27T09:23:13Z)
• Compositional Generative Inverse Design [69.22782875567547]
  Inverse design, where we seek to design input variables in order to optimize an underlying objective function, is an important problem. We show that by instead optimizing over the learned energy function captured by the diffusion model, we can avoid such adversarial examples. In an N-body interaction task and a challenging 2D multi-airfoil design task, we demonstrate that by composing the learned diffusion model at test time, our method allows us to design initial states and boundary shapes.
  arXiv  Detail & Related papers  (2024-01-24T01:33:39Z)
• ECoDe: A Sample-Efficient Method for Co-Design of Robotic Agents [11.449817465618658]
  Co-designing autonomous robotic agents involves simultaneously optimizing the controller and physical design of the agent. This can be challenging when the design space is large. We propose a multi-fidelity-based exploration strategy to improve the sample efficiency of co-design.
  arXiv  Detail & Related papers  (2023-09-08T02:54:31Z)
• Performance Comparison of Design Optimization and Deep Learning-based Inverse Design [15.620304857903069]
  This paper compares the performance of traditional design optimization methods with deep learning-based inverse design methods. It provides guidelines that can be taken into account for the future utilization of deep learning-based inverse design.
  arXiv  Detail & Related papers  (2023-08-23T06:04:54Z)
• Fusion of ML with numerical simulation for optimized propeller design [0.6767885381740952]
  We propose an alternative way to use ML model to surrogate the design process. By using this trained surrogate model with the traditional optimization method, we can get the best of both worlds. Empirical evaluations of propeller design problems show that a better efficient design can be found in fewer evaluations using SAO.
  arXiv  Detail & Related papers  (2023-02-28T16:42:07Z)
• Transfer Learning for Bayesian Optimization: A Survey [29.229660973338145]
  Black-box optimization is a powerful tool that models and optimize such expensive "black-box" functions. Researchers in the BO community propose to incorporate the spirit of transfer learning to accelerate optimization process.
  arXiv  Detail & Related papers  (2023-02-12T14:37:25Z)
• Conservative Objective Models for Effective Offline Model-Based Optimization [78.19085445065845]
  Computational design problems arise in a number of settings, from synthetic biology to computer architectures. We propose a method that learns a model of the objective function that lower bounds the actual value of the ground-truth objective on out-of-distribution inputs. COMs are simple to implement and outperform a number of existing methods on a wide range of MBO problems.
  arXiv  Detail & Related papers  (2021-07-14T17:55:28Z)
• Machine Learning Framework for Quantum Sampling of Highly-Constrained, Continuous Optimization Problems [101.18253437732933]
  We develop a generic, machine learning-based framework for mapping continuous-space inverse design problems into surrogate unconstrained binary optimization problems. We showcase the framework's performance on two inverse design problems by optimizing thermal emitter topologies for thermophotovoltaic applications and (ii) diffractive meta-gratings for highly efficient beam steering.
  arXiv  Detail & Related papers  (2021-05-06T02:22:23Z)
• Generative Design by Reinforcement Learning: Enhancing the Diversity of Topology Optimization Designs [5.8010446129208155]
  This study proposes a reinforcement learning based generative design process, with reward functions maximizing the diversity of topology designs. We show that RL-based generative design produces a large number of diverse designs within a short inference time by exploiting GPU in a fully automated manner.
  arXiv  Detail & Related papers  (2020-08-17T06:50:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.