MetaBox-v2: A Unified Benchmark Platform for Meta-Black-Box Optimization

• URL: http://arxiv.org/abs/2505.17745v1
• Date: Fri, 23 May 2025 11:13:10 GMT
• Title: MetaBox-v2: A Unified Benchmark Platform for Meta-Black-Box Optimization
• Authors: Zeyuan Ma, Yue-Jiao Gong, Hongshu Guo, Wenjie Qiu, Sijie Ma, Hongqiao Lian, Jiajun Zhan, Kaixu Chen, Chen Wang, Zhiyang Huang, Zechuan Huang, Guojun Peng, Ran Cheng, Yining Ma,
• Abstract summary: We introduce MetaBox-v2 as a milestone upgrade with four novel features.<n>A comprehensive benchmark suite of 18 synthetic/realistic tasks spanning single-objective, multi-objective, multi-model, and multi-task optimization scenarios.<n> Valuable insights are concluded from thorough and detailed analysis for practitioners and those new to the field.
• Score: 17.36826574167763
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Meta-Black-Box Optimization (MetaBBO) streamlines the automation of optimization algorithm design through meta-learning. It typically employs a bi-level structure: the meta-level policy undergoes meta-training to reduce the manual effort required in developing algorithms for low-level optimization tasks. The original MetaBox (2023) provided the first open-source framework for reinforcement learning-based single-objective MetaBBO. However, its relatively narrow scope no longer keep pace with the swift advancement in this field. In this paper, we introduce MetaBox-v2 (https://github.com/MetaEvo/MetaBox) as a milestone upgrade with four novel features: 1) a unified architecture supporting RL, evolutionary, and gradient-based approaches, by which we reproduce 23 up-to-date baselines; 2) efficient parallelization schemes, which reduce the training/testing time by 10-40x; 3) a comprehensive benchmark suite of 18 synthetic/realistic tasks (1900+ instances) spanning single-objective, multi-objective, multi-model, and multi-task optimization scenarios; 4) plentiful and extensible interfaces for custom analysis/visualization and integrating to external optimization tools/benchmarks. To show the utility of MetaBox-v2, we carry out a systematic case study that evaluates the built-in baselines in terms of the optimization performance, generalization ability and learning efficiency. Valuable insights are concluded from thorough and detailed analysis for practitioners and those new to the field.

Related papers

• PlatMetaX: An Integrated MATLAB platform for Meta-Black-Box Optimization [13.141855165689448]
  We present PlatMetaX, a novel platform for developing, evaluating, and comparing optimization algorithms.<n>PlatMetaX integrates the strengths of MetaBox and PlatEMO, offering a comprehensive framework for developing, evaluating, and comparing optimization algorithms.
  arXiv  Detail & Related papers  (2025-03-26T02:03:35Z)
• Learning Versatile Optimizers on a Compute Diet [20.69804303768643]
  Key elements in learned architectures and meta-training procedures can lead to strong meta-generalization.<n>We propose evaluation metrics to reliably assess quantitative performance of an at scale on a set of evaluation tasks.<n>Our proposed approach, Celo, makes a significant leap in improving the meta-generalization performance of learneds.
  arXiv  Detail & Related papers  (2025-01-22T06:10:27Z)
• Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers [108.72225067368592]
  We propose a novel perspective to investigate the design of large language models (LLMs)-based prompts.<n>We identify two pivotal factors in model parameter learning: update direction and update method.<n>We develop a capable Gradient-inspired Prompt-based GPO.
  arXiv  Detail & Related papers  (2024-02-27T15:05:32Z)
• MetaBox: A Benchmark Platform for Meta-Black-Box Optimization with Reinforcement Learning [25.687304354503148]
  We introduce MetaBox, the first benchmark platform specifically tailored for developing and evaluating MetaBBO-RL methods. MetaBox offers a flexible algorithmic template that allows users to effortlessly implement their unique designs within the platform. It provides a broad spectrum of over 300 problem instances, collected from synthetic to realistic scenarios, and an extensive library of 19 baseline methods.
  arXiv  Detail & Related papers  (2023-10-12T11:55:17Z)
• Eliminating Meta Optimization Through Self-Referential Meta Learning [5.584060970507506]
  We investigate self-referential meta learning systems that modify themselves without the need for explicit meta optimization. A neural network self-modifies to solve bandit and classic control tasks, improves its self-modifications, and learns how to learn.
  arXiv  Detail & Related papers  (2022-12-29T17:53:40Z)
• Meta Mirror Descent: Optimiser Learning for Fast Convergence [85.98034682899855]
  We take a different perspective starting from mirror descent rather than gradient descent, and meta-learning the corresponding Bregman divergence. Within this paradigm, we formalise a novel meta-learning objective of minimising the regret bound of learning. Unlike many meta-learned optimisers, it also supports convergence and generalisation guarantees and uniquely does so without requiring validation data.
  arXiv  Detail & Related papers  (2022-03-05T11:41:13Z)
• Meta-Learning with Neural Tangent Kernels [58.06951624702086]
  We propose the first meta-learning paradigm in the Reproducing Kernel Hilbert Space (RKHS) induced by the meta-model's Neural Tangent Kernel (NTK) Within this paradigm, we introduce two meta-learning algorithms, which no longer need a sub-optimal iterative inner-loop adaptation as in the MAML framework. We achieve this goal by 1) replacing the adaptation with a fast-adaptive regularizer in the RKHS; and 2) solving the adaptation analytically based on the NTK theory.
  arXiv  Detail & Related papers  (2021-02-07T20:53:23Z)
• Fast Few-Shot Classification by Few-Iteration Meta-Learning [173.32497326674775]
  We introduce a fast optimization-based meta-learning method for few-shot classification. Our strategy enables important aspects of the base learner objective to be learned during meta-training. We perform a comprehensive experimental analysis, demonstrating the speed and effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-10-01T15:59:31Z)
• BOML: A Modularized Bilevel Optimization Library in Python for Meta Learning [52.90643948602659]
  BOML is a modularized optimization library that unifies several meta-learning algorithms into a common bilevel optimization framework. It provides a hierarchical optimization pipeline together with a variety of iteration modules, which can be used to solve the mainstream categories of meta-learning methods.
  arXiv  Detail & Related papers  (2020-09-28T14:21:55Z)
• Guarantees for Tuning the Step Size using a Learning-to-Learn Approach [18.838453594698166]
  We give meta-optimization guarantees for the learning-to-learn approach on a simple problem of tuning the step size for quadratic loss. Although there is a way to design the meta-objective so that the meta-gradient remains bounded, computing the meta-gradient directly using backpropagation leads to numerical issues. We also characterize when it is necessary to compute the meta-objective on a separate validation set to ensure the performance of the learned.
  arXiv  Detail & Related papers  (2020-06-30T02:59:35Z)
• On the Global Optimality of Model-Agnostic Meta-Learning [133.16370011229776]
  Model-a meta-learning (MAML) formulates meta-learning as a bilevel optimization problem, where the inner level solves each subtask based on a shared prior. We characterize optimality of the stationary points attained by MAML for both learning and supervised learning, where the inner-level outer-level problems are solved via first-order optimization methods.
  arXiv  Detail & Related papers  (2020-06-23T17:33:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.