Make Optimization Once and for All with Fine-grained Guidance

• URL: http://arxiv.org/abs/2503.11462v1
• Date: Fri, 14 Mar 2025 14:48:12 GMT
• Title: Make Optimization Once and for All with Fine-grained Guidance
• Authors: Mingjia Shi, Ruihan Lin, Xuxi Chen, Yuhao Zhou, Zezhen Ding, Pingzhi Li, Tong Wang, Kai Wang, Zhangyang Wang, Jiheng Zhang, Tianlong Chen,
• Abstract summary: Learning to Optimize (L2O) enhances optimization efficiency with integrated neural networks.<n>L2O paradigms achieve great outcomes, e.g., refitting, generating unseen solutions iteratively or directly.<n>Our analyses explore general framework for learning optimization, called Diff-L2O, focusing on augmenting solutions from a wider view.
• Score: 78.14885351827232
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Learning to Optimize (L2O) enhances optimization efficiency with integrated neural networks. L2O paradigms achieve great outcomes, e.g., refitting optimizer, generating unseen solutions iteratively or directly. However, conventional L2O methods require intricate design and rely on specific optimization processes, limiting scalability and generalization. Our analyses explore general framework for learning optimization, called Diff-L2O, focusing on augmenting sampled solutions from a wider view rather than local updates in real optimization process only. Meanwhile, we give the related generalization bound, showing that the sample diversity of Diff-L2O brings better performance. This bound can be simply applied to other fields, discussing diversity, mean-variance, and different tasks. Diff-L2O's strong compatibility is empirically verified with only minute-level training, comparing with other hour-levels.

Related papers

• Large Scale Multi-Task Bayesian Optimization with Large Language Models [29.12351845364205]
  We introduce a novel approach leveraging large language models (LLMs) to learn from, and improve upon, previous optimization trajectories.<n>We evaluate our method on two distinct domains: database query optimization and antimicrobial peptide design.
  arXiv  Detail & Related papers  (2025-03-11T07:46:19Z)
• Provably Faster Algorithms for Bilevel Optimization via Without-Replacement Sampling [96.47086913559289]
  gradient-based algorithms are widely used in bilevel optimization. We introduce a without-replacement sampling based algorithm which achieves a faster convergence rate. We validate our algorithms over both synthetic and real-world applications.
  arXiv  Detail & Related papers  (2024-11-07T17:05:31Z)
• Evolutionary Solution Adaption for Multi-Objective Metal Cutting Process Optimization [59.45414406974091]
  We introduce a framework for system flexibility that allows us to study the ability of an algorithm to transfer solutions from previous optimization tasks. We study the flexibility of NSGA-II, which we extend by two variants: 1) varying goals, that optimize solutions for two tasks simultaneously to obtain in-between source solutions expected to be more adaptable, and 2) active-inactive genotype, that accommodates different possibilities that can be activated or deactivated. Results show that adaption with standard NSGA-II greatly reduces the number of evaluations required for optimization to a target goal, while the proposed variants further improve the adaption costs.
  arXiv  Detail & Related papers  (2023-05-31T12:07:50Z)
• M-L2O: Towards Generalizable Learning-to-Optimize by Test-Time Fast Self-Adaptation [145.7321032755538]
  Learning to Optimize (L2O) has drawn increasing attention as it often remarkably accelerates the optimization procedure of complex tasks. This paper investigates a potential solution to this open challenge by meta-training an L2O that can perform fast test-time self-adaptation to an out-of-distribution task.
  arXiv  Detail & Related papers  (2023-02-28T19:23:20Z)
• Learning to Generalize Provably in Learning to Optimize [185.71326306329678]
  Learning to optimize (L2O) has gained increasing popularity, which automates the design of optimizees by data-driven approaches. Current L2O methods often suffer from poor generalization performance in at least two folds. We propose to incorporate these two metrics as flatness-aware regularizers into the L2O framework.
  arXiv  Detail & Related papers  (2023-02-22T01:17:31Z)
• Teaching Networks to Solve Optimization Problems [13.803078209630444]
  We propose to replace the iterative solvers altogether with a trainable parametric set function. We show the feasibility of learning such parametric (set) functions to solve various classic optimization problems.
  arXiv  Detail & Related papers  (2022-02-08T19:13:13Z)
• Learning to Optimize: A Primer and A Benchmark [94.29436694770953]
  Learning to optimize (L2O) is an emerging approach that leverages machine learning to develop optimization methods. This article is poised to be the first comprehensive survey and benchmark of L2O for continuous optimization.
  arXiv  Detail & Related papers  (2021-03-23T20:46:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.