Information-Theoretic Bayesian Optimization for Bilevel Optimization Problems

• URL: http://arxiv.org/abs/2509.21725v1
• Date: Fri, 26 Sep 2025 00:52:14 GMT
• Title: Information-Theoretic Bayesian Optimization for Bilevel Optimization Problems
• Authors: Takuya Kanayama, Yuki Ito, Tomoyuki Tamura, Masayuki Karasuyama,
• Abstract summary: A bilevel optimization problem consists of two optimization problems nested as an upper- and a lower-level problem.<n>We propose an information-theoretic approach that considers the information gain of both the upper- and lower-optimal solutions and values.<n>We empirically demonstrate the effectiveness of our proposed method through several benchmark datasets.
• Score: 8.423854716028094
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A bilevel optimization problem consists of two optimization problems nested as an upper- and a lower-level problem, in which the optimality of the lower-level problem defines a constraint for the upper-level problem. This paper considers Bayesian optimization (BO) for the case that both the upper- and lower-levels involve expensive black-box functions. Because of its nested structure, bilevel optimization has a complex problem definition and, compared with other standard extensions of BO such as multi-objective or constraint settings, it has not been widely studied. We propose an information-theoretic approach that considers the information gain of both the upper- and lower-optimal solutions and values. This enables us to define a unified criterion that measures the benefit for both level problems, simultaneously. Further, we also show a practical lower bound based approach to evaluating the information gain. We empirically demonstrate the effectiveness of our proposed method through several benchmark datasets.

Related papers

• BILBO: BILevel Bayesian Optimization [36.88993287853141]
  Bilevel optimization is characterized by a two-level structure, where the upper-level problem is constrained by optimal lower-level solutions.<n>We present BILevel Bayesian Optimization (BILBO), a novel algorithm for general bilevel problems with blackbox functions.
  arXiv  Detail & Related papers  (2025-02-04T08:57:47Z)
• Safe Gradient Flow for Bilevel Optimization [6.3963012138521975]
  Bilevel optimization is a key framework in hierarchical decision-making.<n>We propose a control-theoretic approach to solving bilevel optimization problems.
  arXiv  Detail & Related papers  (2025-01-27T21:39:25Z)
• Bayesian Optimization of Bilevel Problems [0.0]
  This paper focuses on bilevel optimization where both upper-level and lower-level functions are black boxes and expensive to evaluate.<n>We propose a Bayesian Optimization framework that models the upper and lower-level functions as Gaussian processes over the combined space of upper and lower-level decisions.<n>Our experimental results demonstrate that the proposed algorithm is highly sample-efficient and outperforms existing methods in finding high-quality solutions.
  arXiv  Detail & Related papers  (2024-12-24T15:55:30Z)
• Provably Faster Algorithms for Bilevel Optimization via Without-Replacement Sampling [96.47086913559289]
  gradient-based algorithms are widely used in bilevel optimization. We introduce a without-replacement sampling based algorithm which achieves a faster convergence rate. We validate our algorithms over both synthetic and real-world applications.
  arXiv  Detail & Related papers  (2024-11-07T17:05:31Z)
• Pareto Set Prediction Assisted Bilevel Multi-objective Optimization [2.3293561091456283]
  We address problems with multiple objectives (BLMOP) at both levels. The proposed approach is competitive across a range of problems, including both deceptive and non-deceptive.
  arXiv  Detail & Related papers  (2024-09-05T08:04:11Z)
• A Primal-Dual-Assisted Penalty Approach to Bilevel Optimization with Coupled Constraints [66.61399765513383]
  We develop a BLOCC algorithm to tackle BiLevel Optimization problems with Coupled Constraints. We demonstrate its effectiveness on two well-known real-world applications.
  arXiv  Detail & Related papers  (2024-06-14T15:59:36Z)
• Contextual Stochastic Bilevel Optimization [50.36775806399861]
  We introduce contextual bilevel optimization (CSBO) -- a bilevel optimization framework with the lower-level problem minimizing an expectation on some contextual information and the upper-level variable. It is important for applications such as meta-learning, personalized learning, end-to-end learning, and Wasserstein distributionally robustly optimization with side information (WDRO-SI)
  arXiv  Detail & Related papers  (2023-10-27T23:24:37Z)
• Effective Bilevel Optimization via Minimax Reformulation [23.5093932552053]
  We propose a reformulation of bilevel optimization as a minimax problem. Under mild conditions, we show these two problems are equivalent. Our method outperforms state-of-the-art bilevel methods while significantly reducing the computational cost.
  arXiv  Detail & Related papers  (2023-05-22T15:41:33Z)
• Communication-Efficient Robust Federated Learning with Noisy Labels [144.31995882209932]
  Federated learning (FL) is a promising privacy-preserving machine learning paradigm over distributed located data. We propose a learning-based reweighting approach to mitigate the effect of noisy labels in FL. Our approach has shown superior performance on several real-world datasets compared to various baselines.
  arXiv  Detail & Related papers  (2022-06-11T16:21:17Z)
• A Constrained Optimization Approach to Bilevel Optimization with Multiple Inner Minima [49.320758794766185]
  We propose a new approach, which convert the bilevel problem to an equivalent constrained optimization, and then the primal-dual algorithm can be used to solve the problem. Such an approach enjoys a few advantages including (a) addresses the multiple inner minima challenge; (b) fully first-order efficiency without Jacobian computations.
  arXiv  Detail & Related papers  (2022-03-01T18:20:01Z)
• Enhanced Bilevel Optimization via Bregman Distance [104.96004056928474]
  We propose a bilevel optimization method based on Bregman Bregman functions. We also propose an accelerated version of SBiO-BreD method (ASBiO-BreD) by using the variance-reduced technique.
  arXiv  Detail & Related papers  (2021-07-26T16:18:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.