Relation Reasoning with LLMs in Expensive Optimization

• URL: http://arxiv.org/abs/2605.02933v1
• Date: Thu, 30 Apr 2026 04:28:10 GMT
• Title: Relation Reasoning with LLMs in Expensive Optimization
• Abstract summary: This paper proposes R2SAEA, a reinforcement-trained relation-based large language model (LLM) surrogate assisted evolutionary algorithm.<n>We cast relation-based surrogate modeling as an in-context pairwise reasoning task.<n> Experiments on single- and multi-objective benchmarks show improved relation prediction and state-of-the-art optimization performance.
• Score: 19.271995977675758
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Expensive optimization problems (EOPs) are black-box tasks with costly objective evaluations and no gradient access, making the evaluation budget the key bottleneck. Surrogate-assisted evolutionary algorithms (SAEAs) reduce evaluations via surrogate predictions, but conventional surrogates often require frequent retraining as populations evolve, incurring overhead. This paper proposes R2SAEA, a reinforcement-trained relation-based large language model (LLM) surrogate assisted evolutionary algorithm. We cast relation-based surrogate modeling as an in-context pairwise reasoning task. To enable efficient inference in evolutionary loops, we develop an anchor-based iterative context construction strategy that reduces prompt complexity from quadratic to linear in population size, and a voting-based aggregation scheme that converts predicted relations into scores for offspring selection. We further build an RL pipeline from evolutionary trajectories and fine-tune Qwen2.5 with GRPO. Experiments on single- and multi-objective benchmarks show improved relation prediction and state-of-the-art optimization performance over strong SAEA baselines and general LLMs. Quantization also enables efficient edge deployment, supporting a zero-shot surrogate paradigm without per-generation retraining. Code and models are available at https://github.com/Septend9/R2SAEA.

Related papers

• In-Context Learning for Latent Space Bayesian Optimization [8.943469192703875]
  latent-space Bayesian optimization (LSBO) extends it to structured objects such as molecules and proteins.<n>LSBO creates a distinctive mismatch: the induced map from latent code to objective value differs markedly from the regression tasks used to train current in-context models.<n>We address this mismatch by complementing the pretraining stage of tabular foundation model surrogates with synthetic optimization tasks defined on the latent space of a molecular VAE.
  arXiv  Detail & Related papers  (2026-06-08T15:45:25Z)
• Programmatic Context Augmentation for LLM-based Symbolic Regression [65.01826333382738]
  Large language model (LLM)-based evolutionary search methods have been introduced into Symbolic regression.<n>We propose a novel LLM-based evolutionary search framework that incorporates programmatic context augmentation.<n>By enabling code-based interactions with the dataset, our method can actively perform data analysis and extract informative signals.
  arXiv  Detail & Related papers  (2026-05-04T19:34:03Z)
• ReVEL: Multi-Turn Reflective LLM-Guided Heuristic Evolution via Structured Performance Feedback [0.0]
  We propose ReVEL: Multi-Turn Reflective LLM-Heuristic Evolution via Structured Performance Feedback.<n>The core of ReVEL lies in two mechanisms: (i) performance-profile grouping and (ii) multi-turn feedback-driven reflection.<n>Our results highlight multi-turn reasoning with structured grouping as a principled paradigm for automated design.
  arXiv  Detail & Related papers  (2026-03-05T04:52:11Z)
• MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization [56.074760766965085]
  Group-Relative Policy Optimization has emerged as an efficient paradigm for aligning Large Language Models (LLMs)<n>We propose MAESTRO, which treats reward scalarization as a dynamic latent policy, leveraging the model's terminal hidden states as a semantic bottleneck.<n>We formulate this as a contextual bandit problem within a bi-level optimization framework, where a lightweight Conductor network co-evolves with the policy by utilizing group-relative advantages as a meta-reward signal.
  arXiv  Detail & Related papers  (2026-01-12T05:02:48Z)
• Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective [85.06838178922791]
  Reinforcement Learning (RL) has proven highly effective for autoregressive language models.<n>But adapting these methods to diffusion large language models (dLLMs) presents fundamental challenges.<n>We propose a principled RL framework that treats entire sequence generation as a single action and uses the ELBO as a tractable sequence-level likelihood proxy.
  arXiv  Detail & Related papers  (2025-12-03T13:05:32Z)
• Agentic Reinforced Policy Optimization [66.96989268893932]
  Large-scale reinforcement learning with verifiable rewards (RLVR) has demonstrated its effectiveness in harnessing the potential of large language models (LLMs) for single-turn reasoning tasks.<n>Current RL algorithms inadequately balance the models' intrinsic long-horizon reasoning capabilities and their proficiency in multi-turn tool interactions.<n>We propose Agentic Reinforced Policy Optimization (ARPO), a novel agentic RL algorithm tailored for training multi-turn LLM-based agents.
  arXiv  Detail & Related papers  (2025-07-26T07:53:11Z)
• Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing [58.52119063742121]
  Retraining a model using its own predictions together with the original, potentially noisy labels is a well-known strategy for improving the model performance.<n>This paper addresses the question of how to optimally combine the model's predictions and the provided labels.<n>Our main contribution is the derivation of the Bayes optimal aggregator function to combine the current model's predictions and the given labels.
  arXiv  Detail & Related papers  (2025-05-21T07:16:44Z)
• Robust Reinforcement Learning from Human Feedback for Large Language Models Fine-Tuning [11.31665596884142]
  Reinforcement learning from human feedback (RLHF) has emerged as a key technique for aligning the output of large language models with human preferences.<n>Most existing RLHF algorithms use the Bradley-Terry model, which relies on assumptions about human preferences that may not reflect the complexity and variability of real-world judgments.<n>We propose a robust algorithm to enhance the performance of existing approaches under such reward model misspecifications.
  arXiv  Detail & Related papers  (2025-04-03T16:16:35Z)
• Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization [75.1240295759264]
  We propose an effective framework for Bridging and Modeling Correlations in pairwise data, named BMC.<n>We increase the consistency and informativeness of the pairwise preference signals through targeted modifications.<n>We identify that DPO alone is insufficient to model these correlations and capture nuanced variations.
  arXiv  Detail & Related papers  (2024-08-14T11:29:47Z)
• A First Look at Kolmogorov-Arnold Networks in Surrogate-assisted Evolutionary Algorithms [5.198324938447394]
  Surrogate-assisted Evolutionary Algorithm (SAEA) is an essential method for solving expensive problems. This paper introduces Kolmogorov-Arnold Networks (KANs) as surrogate models within SAEAs. KANs demonstrate commendable performance within SAEAs, effectively decreasing the number of function calls and enhancing the optimization efficiency.
  arXiv  Detail & Related papers  (2024-05-26T09:12:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.