LLaMEA-SAGE: Guiding Automated Algorithm Design with Structural Feedback from Explainable AI

• URL: http://arxiv.org/abs/2601.21511v1
• Date: Thu, 29 Jan 2026 10:27:29 GMT
• Title: LLaMEA-SAGE: Guiding Automated Algorithm Design with Structural Feedback from Explainable AI
• Authors: Niki van Stein, Anna V. Kononova, Lars Kotthoff, Thomas Bäck,
• Abstract summary: Large language models have enabled automated algorithm design (AAD) by generating optimization algorithms directly from natural-language prompts.<n>We propose a mechanism for guiding AAD using feedback constructed from graph-theoretic and complexity features extracted from the abstract syntax trees of the generated algorithms.<n>We show that the proposed structured guidance achieves the same performance faster than vanilla LLaMEA in a small controlled experiment.
• Score: 4.440668887299803
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models have enabled automated algorithm design (AAD) by generating optimization algorithms directly from natural-language prompts. While evolutionary frameworks such as LLaMEA demonstrate strong exploratory capabilities across the algorithm design space, their search dynamics are entirely driven by fitness feedback, leaving substantial information about the generated code unused. We propose a mechanism for guiding AAD using feedback constructed from graph-theoretic and complexity features extracted from the abstract syntax trees of the generated algorithms, based on a surrogate model learned over an archive of evaluated solutions. Using explainable AI techniques, we identify features that substantially affect performance and translate them into natural-language mutation instructions that steer subsequent LLM-based code generation without restricting expressivity. We propose LLaMEA-SAGE, which integrates this feature-driven guidance into LLaMEA, and evaluate it across several benchmarks. We show that the proposed structured guidance achieves the same performance faster than vanilla LLaMEA in a small controlled experiment. In a larger-scale experiment using the MA-BBOB suite from the GECCO-MA-BBOB competition, our guided approach achieves superior performance compared to state-of-the-art AAD methods. These results demonstrate that signals derived from code can effectively bias LLM-driven algorithm evolution, bridging the gap between code structure and human-understandable performance feedback in automated algorithm design.

Related papers

• From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priors [4.253872963674906]
  Large Language Models (LLMs) have been widely adopted for automated algorithm design.<n>We show that providing high-quality algorithmic code examples can substantially improve the performance of the LLM-driven optimization.
  arXiv  Detail & Related papers  (2026-03-03T09:27:52Z)
• From Performance to Understanding: A Vision for Explainable Automated Algorithm Design [2.780126715305372]
  This paper argues that the next breakthrough will come not from more automation, but from coupling automation with understanding.<n>We outline a vision for explainable automated algorithm design, built on three pillars.<n>These elements form a closed knowledge loop in which discovery, explanation and generalisation reinforce each other.
  arXiv  Detail & Related papers  (2025-11-20T10:08:40Z)
• Experience-Guided Reflective Co-Evolution of Prompts and Heuristics for Automatic Algorithm Design [124.54166764570972]
  Combinatorial optimization problems are traditionally tackled with handcrafted algorithms.<n>Recent progress has highlighted the potential of automatics design powered by large language models.<n>We propose the Experience-Evolution Reflective Co-Guided of Prompt and Heuristics (EvoPH) for automatic algorithm design.
  arXiv  Detail & Related papers  (2025-09-29T09:24:09Z)
• LLM4CMO: Large Language Model-aided Algorithm Design for Constrained Multiobjective Optimization [54.35609820607923]
  Large language models (LLMs) offer new opportunities for assisting with algorithm design.<n>We propose LLM4CMO, a novel CMOEA based on a dual-population, two-stage framework.<n>LLMs can serve as efficient co-designers in the development of complex evolutionary optimization algorithms.
  arXiv  Detail & Related papers  (2025-08-16T02:00:57Z)
• Agentic Reinforced Policy Optimization [66.96989268893932]
  Large-scale reinforcement learning with verifiable rewards (RLVR) has demonstrated its effectiveness in harnessing the potential of large language models (LLMs) for single-turn reasoning tasks.<n>Current RL algorithms inadequately balance the models' intrinsic long-horizon reasoning capabilities and their proficiency in multi-turn tool interactions.<n>We propose Agentic Reinforced Policy Optimization (ARPO), a novel agentic RL algorithm tailored for training multi-turn LLM-based agents.
  arXiv  Detail & Related papers  (2025-07-26T07:53:11Z)
• Behaviour Space Analysis of LLM-driven Meta-heuristic Discovery [2.0958996366233094]
  We investigate meta-heuristic optimisation algorithms automatically generated by Large Language Model driven algorithm discovery methods.<n>We iteratively evolve black-box optimisations, evaluated on 10 functions from the BBOB benchmark suite.<n>We log behavioural metrics including exploration, exploitation, convergence and stagnation measures, for each run, and analyse these via visual projections and network-based representations.
  arXiv  Detail & Related papers  (2025-07-04T14:19:39Z)
• BLADE: Benchmark suite for LLM-driven Automated Design and Evolution of iterative optimisation heuristics [2.2485774453793037]
  BLADE is a framework for benchmarking LLM-driven AAD methods in a continuous black-box optimisation context.<n>It integrates benchmark problems with instance generators and textual descriptions aimed at capability-focused testing, such as specialisation and information exploitation.<n> BLADE provides an out-of-the-box' solution to systematically evaluate LLM-driven AAD approaches.
  arXiv  Detail & Related papers  (2025-04-28T18:34:09Z)
• EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration [76.66831821738927]
  Large language models (LLMs) remain under-studied in scenarios requiring optimal decision-making under uncertainty.<n>We measure LLMs' (in)ability to make optimal decisions in bandits, a state-less reinforcement learning setting relevant to many applications.<n>Motivated by the existence of optimal exploration algorithms, we propose efficient ways to integrate this algorithmic knowledge into LLMs.
  arXiv  Detail & Related papers  (2024-10-08T17:54:03Z)
• Designing Algorithms Empowered by Language Models: An Analytical Framework, Case Studies, and Insights [86.06371692309972]
  This work presents an analytical framework for the design and analysis of large language models (LLMs)-based algorithms.<n>Our proposed framework serves as an attempt to mitigate such headaches.
  arXiv  Detail & Related papers  (2024-07-20T07:39:07Z)
• LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics [0.023020018305241332]
  This paper introduces a novel Large Language Model Evolutionary Algorithm (LLaMEA) framework.<n>Given a set of criteria and a task definition (the search space), LLaMEA iteratively generates, mutates and selects algorithms.<n>We show how this framework can be used to generate novel black-box metaheuristic optimization algorithms automatically.
  arXiv  Detail & Related papers  (2024-05-30T15:10:59Z)
• Accelerating LLaMA Inference by Enabling Intermediate Layer Decoding via Instruction Tuning with LITE [62.13435256279566]
  Large Language Models (LLMs) have achieved remarkable performance across a wide variety of natural language tasks. However, their large size makes their inference slow and computationally expensive. We show that it enables these layers to acquire 'good' generation ability without affecting the generation ability of the final layer.
  arXiv  Detail & Related papers  (2023-10-28T04:07:58Z)
• Improved Algorithms for Neural Active Learning [74.89097665112621]
  We improve the theoretical and empirical performance of neural-network(NN)-based active learning algorithms for the non-parametric streaming setting. We introduce two regret metrics by minimizing the population loss that are more suitable in active learning than the one used in state-of-the-art (SOTA) related work.
  arXiv  Detail & Related papers  (2022-10-02T05:03:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.