Guided Trajectory Generation with Diffusion Models for Offline Model-based Optimization

• URL: http://arxiv.org/abs/2407.01624v1
• Date: Sat, 29 Jun 2024 06:12:36 GMT
• Title: Guided Trajectory Generation with Diffusion Models for Offline Model-based Optimization
• Authors: Taeyoung Yun, Sujin Yun, Jaewoo Lee, Jinkyoo Park,
• Abstract summary: We introduce a novel conditional generative modeling approach to produce trajectories toward high-scoring regions. Experiment results demonstrate that our method outperforms competitive baselines on Design-Bench.
• Score: 19.228156994045587
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Optimizing complex and high-dimensional black-box functions is ubiquitous in science and engineering fields. Unfortunately, the online evaluation of these functions is restricted due to time and safety constraints in most cases. In offline model-based optimization (MBO), we aim to find a design that maximizes the target function using only a pre-existing offline dataset. While prior methods consider forward or inverse approaches to address the problem, these approaches are limited by conservatism and the difficulty of learning highly multi-modal mappings. Recently, there has been an emerging paradigm of learning to improve solutions with synthetic trajectories constructed from the offline dataset. In this paper, we introduce a novel conditional generative modeling approach to produce trajectories toward high-scoring regions. First, we construct synthetic trajectories toward high-scoring regions using the dataset while injecting locality bias for consistent improvement directions. Then, we train a conditional diffusion model to generate trajectories conditioned on their scores. Lastly, we sample multiple trajectories from the trained model with guidance to explore high-scoring regions beyond the dataset and select high-fidelity designs among generated trajectories with the proxy function. Extensive experiment results demonstrate that our method outperforms competitive baselines on Design-Bench and its practical variants. The code is publicly available in \texttt{https://github.com/dbsxodud-11/GTG}.

Related papers

• Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decoding [84.3224556294803]
  Diffusion models excel at capturing the natural design spaces of images, molecules, DNA, RNA, and protein sequences. We aim to optimize downstream reward functions while preserving the naturalness of these design spaces. Our algorithm integrates soft value functions, which looks ahead to how intermediate noisy states lead to high rewards in the future.
  arXiv  Detail & Related papers  (2024-08-15T16:47:59Z)
• Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models [54.132297393662654]
  We introduce a hybrid method that fine-tunes cutting-edge diffusion models by optimizing reward models through RL. We demonstrate the capability of our approach to outperform the best designs in offline data, leveraging the extrapolation capabilities of reward models.
  arXiv  Detail & Related papers  (2024-05-30T03:57:29Z)
• Functional Graphical Models: Structure Enables Offline Data-Driven Optimization [111.28605744661638]
  We show how structure can enable sample-efficient data-driven optimization. We also present a data-driven optimization algorithm that infers the FGM structure itself.
  arXiv  Detail & Related papers  (2024-01-08T22:33:14Z)
• ROMO: Retrieval-enhanced Offline Model-based Optimization [14.277672372460785]
  Data-driven black-box model-based optimization (MBO) problems arise in a number of practical application scenarios. We propose retrieval-enhanced offline model-based optimization (ROMO) ROMO is simple to implement and outperforms state-of-the-art approaches in the CoMBO setting.
  arXiv  Detail & Related papers  (2023-10-11T15:04:33Z)
• Aligning Optimization Trajectories with Diffusion Models for Constrained Design Generation [17.164961143132473]
  We introduce a learning framework that demonstrates the efficacy of aligning the sampling trajectory of diffusion models with the optimization trajectory derived from traditional physics-based methods. Our method allows for generating feasible and high-performance designs in as few as two steps without the need for expensive preprocessing, external surrogate models, or additional labeled data. Our results demonstrate that TA outperforms state-of-the-art deep generative models on in-distribution configurations and halves the inference computational cost.
  arXiv  Detail & Related papers  (2023-05-29T09:16:07Z)
• Robust Model-Based Optimization for Challenging Fitness Landscapes [96.63655543085258]
  Protein design involves optimization on a fitness landscape. Leading methods are challenged by sparsity of high-fitness samples in the training set. We show that this problem of "separation" in the design space is a significant bottleneck in existing model-based optimization tools. We propose a new approach that uses a novel VAE as its search model to overcome the problem.
  arXiv  Detail & Related papers  (2023-05-23T03:47:32Z)
• Conservative Objective Models for Effective Offline Model-Based Optimization [78.19085445065845]
  Computational design problems arise in a number of settings, from synthetic biology to computer architectures. We propose a method that learns a model of the objective function that lower bounds the actual value of the ground-truth objective on out-of-distribution inputs. COMs are simple to implement and outperform a number of existing methods on a wide range of MBO problems.
  arXiv  Detail & Related papers  (2021-07-14T17:55:28Z)
• An end-to-end data-driven optimisation framework for constrained trajectories [4.73357470713202]
  We leverage data-driven approaches to design a new end-to-end framework for optimisation problems. We apply our approach to two settings in aeronautics and sailing routes, yielding commanding results.
  arXiv  Detail & Related papers  (2020-11-24T00:54:17Z)
• Haar Wavelet based Block Autoregressive Flows for Trajectories [129.37479472754083]
  Prediction of trajectories such as that of pedestrians is crucial to the performance of autonomous agents. We introduce a novel Haar wavelet based block autoregressive model leveraging split couplings. We illustrate the advantages of our approach for generating diverse and accurate trajectories on two real-world datasets.
  arXiv  Detail & Related papers  (2020-09-21T13:57:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.