BEAM: Bi-level Memory-adaptive Algorithmic Evolution for LLM-Powered Heuristic Design

• URL: http://arxiv.org/abs/2604.12898v1
• Date: Tue, 14 Apr 2026 15:46:47 GMT
• Title: BEAM: Bi-level Memory-adaptive Algorithmic Evolution for LLM-Powered Heuristic Design
• Abstract summary: Large Language Model-based Hyper Heuristic (LHH) has recently emerged as an efficient way for automatic design.<n>We propose textbfBEAM (Bi-level Memory-adaptive Algorithmic Evolution) to address this problem.<n>We show that BEAM significantly outperforms existing LHHs, notably reducing the optimality gap by 37.84% on aggregate.
• Score: 49.50107918295052
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large Language Model-based Hyper Heuristic (LHH) has recently emerged as an efficient way for automatic heuristic design. However, most existing LHHs just perform well in optimizing a single function within a pre-defined solver. Their single-layer evolution makes them not effective enough to write a competent complete solver. While some variants incorporate hyperparameter tuning or attempt to generate complex code through iterative local modifications, they still lack a high-level algorithmic modeling, leading to limited exploration efficiency. To address this, we reformulate heuristic design as a Bi-level Optimization problem and propose \textbf{BEAM} (Bi-level Memory-adaptive Algorithmic Evolution). BEAM's exterior layer evolves high-level algorithmic structures with function placeholders through genetic algorithm (GA), while the interior layer realizes these placeholders via Monte Carlo Tree Search (MCTS). We further introduce an Adaptive Memory module to facilitate complex code generation. To support the evaluation for complex code generation, we point out the limitations of starting LHHs from scratch or from code templates and introduce a Knowledge Augmentation (KA) Pipeline. Experimental results on several optimization problems demonstrate that BEAM significantly outperforms existing LHHs, notably reducing the optimality gap by 37.84\% on aggregate in CVRP hybrid algorithm design. BEAM also designs a heuristic that outperforms SOTA Maximum Independent Set (MIS) solver KaMIS.

Related papers

• A Hybrid Nested Harness for Decoupling Structure and Parameters in LLM-Driven Optimization [6.599344783327054]
  In evolutionary algorithms powered by language models, the LLM acts as a single operator that simultaneously updates structural components and continuous parameters.<n>We formalize a hybrid nested search, in which an outer loop has the LLM propose a structural sketch, with numeric gaps, and an innernest tunes the sketch.<n>We validate our framework across three scientific domains: (i) meta-optimizers on closed-form test functions, (ii) code-based policies for systems research and social dilemmas, and (iii) approximate Bayesian inference tasks.
  arXiv  Detail & Related papers  (2026-08-08T14:35:22Z)
• FrontierOR: Benchmarking LLMs' Capacity for Efficient Algorithm Design in Large-Scale Optimization [61.43300970020897]
  Large language models (LLMs) are increasingly used for optimization modeling and solver-code generation.<n>Existing benchmarks are limited to small or simplified examples far below real-world scale and complexity.<n>We introduce FrontierOR, among the first benchmarks to evaluate LLM-based efficient algorithm design for realistic large-scale optimization problems.
  arXiv  Detail & Related papers  (2026-05-24T20:10:42Z)
• A2DEPT: Large Language Model-Driven Automated Algorithm Design via Evolutionary Program Trees [8.49373236378493]
  Large Language Model (LLM)-based Automated Heuristic Design (AHD) has shown promise in autonomously generating components with minimal human intervention.<n>To enable open-ended solver beyond rigid templates, we propose Automated Evolutionary Program Trees (A2DEPT)<n>A2DEPT explores the vast program space via a tree-structured evolutionary search with hybrid selection and hierarchical operators, enabling iterative refinement of complete algorithms.
  arXiv  Detail & Related papers  (2026-04-27T05:07:10Z)
• Applying Grover-mixer Quantum Alternating Operator Ansatz Algorithm to High-order Unconstrained Binary Optimization Problems [0.21847754147782883]
  Grover-mixer variant (GM-QAOA) offers compelling alternative due to its global search capabilities.<n>We present a numerical study demonstrating that GM-QAOA, unlike XM-QAOA, exhibits monotonic performance improvement with circuit depth.
  arXiv  Detail & Related papers  (2025-12-28T18:01:20Z)
• BAMBO: Construct Ability and Efficiency LLM Pareto Set via Bayesian Adaptive Multi-objective Block-wise Optimization [4.196004665145396]
  BAMBO (Bayesian Adaptive Multi-objective Block-wise Optimization) is a novel framework that automatically constructs the Large Language Models (LLMs)<n>Formulated as a 1D clustering problem, this strategy leverages a dynamic programming approach to optimally balance intra-blockvolume and inter-block information distribution.
  arXiv  Detail & Related papers  (2025-12-10T15:32:56Z)
• LLM4CMO: Large Language Model-aided Algorithm Design for Constrained Multiobjective Optimization [54.35609820607923]
  Large language models (LLMs) offer new opportunities for assisting with algorithm design.<n>We propose LLM4CMO, a novel CMOEA based on a dual-population, two-stage framework.<n>LLMs can serve as efficient co-designers in the development of complex evolutionary optimization algorithms.
  arXiv  Detail & Related papers  (2025-08-16T02:00:57Z)
• Optimizing Prompt Sequences using Monte Carlo Tree Search for LLM-Based Optimization [20.44067161623662]
  Large language models (LLMs) have demonstrated remarkable capabilities in code generation and structured reasoning.<n>We propose a novel neural-symbolic framework that formulates prompt selection as a sequential decision process guided by Monte Carlo Tree Search.<n>Our method explores and refines multi-step prompt sequences for the goal of improving code generation quality.
  arXiv  Detail & Related papers  (2025-08-08T04:01:24Z)
• Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark [166.40879020706151]
  This paper proposes a shift towards BP-free, zeroth-order (ZO) optimization as a solution for reducing memory costs during fine-tuning. Unlike traditional ZO-SGD methods, our work expands the exploration to a wider array of ZO optimization techniques. Our study unveils previously overlooked optimization principles, highlighting the importance of task alignment, the role of the forward gradient method, and the balance between algorithm complexity and fine-tuning performance.
  arXiv  Detail & Related papers  (2024-02-18T14:08:48Z)
• A Nearly Optimal and Low-Switching Algorithm for Reinforcement Learning with General Function Approximation [67.66904892192794]
  We propose a new algorithm, Monotonic Q-Learning with Upper Confidence Bound (MQL-UCB) for reinforcement learning.<n>MQL-UCB achieves minimax optimal regret of $tildeO(dsqrtHK)$ when $K$ is sufficiently large and near-optimal policy switching cost.<n>Our work sheds light on designing provably sample-efficient and deployment-efficient Q-learning with nonlinear function approximation.
  arXiv  Detail & Related papers  (2023-11-26T08:31:57Z)
• Maximize to Explore: One Objective Function Fusing Estimation, Planning, and Exploration [87.53543137162488]
  We propose an easy-to-implement online reinforcement learning (online RL) framework called textttMEX. textttMEX integrates estimation and planning components while balancing exploration exploitation automatically. It can outperform baselines by a stable margin in various MuJoCo environments with sparse rewards.
  arXiv  Detail & Related papers  (2023-05-29T17:25:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.