Offline Multi-Task Multi-Objective Data-Driven Evolutionary Algorithm with Language Surrogate Model and Implicit Q-Learning

• URL: http://arxiv.org/abs/2512.15149v1
• Date: Wed, 17 Dec 2025 07:30:11 GMT
• Title: Offline Multi-Task Multi-Objective Data-Driven Evolutionary Algorithm with Language Surrogate Model and Implicit Q-Learning
• Authors: Xian-Rong Zhang, Yue-Jiao Gong, Zeyuan Ma, Jun Zhang,
• Abstract summary: We propose Q-MetaSur as a plug-and-play surrogate modeling scheme.<n>We consider multi-task-multi-objective optimization(MTMOO) in offline setting.<n>We show that Q-MetaSur outperforms representative surrogate baselines in objective approximation accuracy.
• Score: 16.52427396199173
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data-driven evolutionary algorithms has shown surprising results in addressing expensive optimization problems through robust surrogate modeling. Though promising, existing surrogate modeling schemes may encounter limitations in complex optimization problems with many sub-objectives, which rely on repeated and tedious approximation. To address such technical gap, we propose Q-MetaSur as a plug-and-play surrogate modeling scheme capable of providing unified and generalized surrogate learning. Specifically, we consider multi-task-multi-objective optimization~(MTMOO) in offline setting. Several key designs are proposed: 1) we transform objective approximation into sequence-to-sequence modeling where MTMOO problem can be represented by tenxual tokenization. To operate under such auto-regressive modeling, we introduce a Large Language Model-based surrogate model that first encodes a MTMOO instance and then decodes objective values of unseen decision variables. To ensure stability in training the proposed model, we propose a two-stage offline training strategy that operates as a synergy of supervised tuning and RL fine-tuning, which first exploits offline dataset to fit existing knowledge and then leverages RL to enhance model's generalization performance. Extensive empirical results on the CEC2019 benchmark demonstrate that Q-MetaSur not only outperforms representative surrogate baselines in objective approximation accuracy, but also helps underlying evolutionary algorithms achieve both desired optimization convergence and improved pareto optimality.

Related papers

• Surrogate Ensemble in Expensive Multi-Objective Optimization via Deep Q-Learning [17.84264663466905]
  Surrogate-assisted Evolutionary Algorithms(SAEAs) have shown promising robustness in solving expensive optimization problems.<n>A key aspect that impacts SAEAs' effectiveness is surrogate model selection, which in existing works is predominantly decided by human developer.<n>We propose a reinforcement learning-assisted ensemble framework, termed as SEEMOO, which is capable of scheduling different surrogate models within a single optimization process.
  arXiv  Detail & Related papers  (2026-01-31T06:14:27Z)
• Automated Optimization Modeling via a Localizable Error-Driven Perspective [20.591721861026414]
  We propose a novel error-driven learning framework for automated optimization modeling.<n>MIND customized the whole model training framework from data synthesis to post-training.<n>MIND consistently outperforms all the state-of-the-art automated optimization modeling approaches.
  arXiv  Detail & Related papers  (2026-01-17T09:59:01Z)
• Merge and Guide: Unifying Model Merging and Guided Decoding for Controllable Multi-Objective Generation [49.98025799046136]
  We introduce Merge-And-GuidE, a two-stage framework that leverages model merging for guided decoding.<n>In Stage 1, MAGE resolves a compatibility problem between the guidance and base models.<n>In Stage 2, we merge explicit and implicit value models into a unified guidance proxy, which then steers the decoding of the base model from Stage 1.
  arXiv  Detail & Related papers  (2025-10-04T11:10:07Z)
• BERT4beam: Large AI Model Enabled Generalized Beamforming Optimization [77.17508487745026]
  This paper investigates the large-scale AI model designed for beamforming optimization to adapt and generalize to diverse tasks defined by system utilities and scales.<n>We propose a novel framework based on bidirectional encoder representations from transformers (BERT), termed BERT4 encoder.<n>Based on the framework, we propose two BERT-based approaches for single-task and multi-task beamforming optimization, respectively.
  arXiv  Detail & Related papers  (2025-09-14T02:49:29Z)
• Leveraging Importance Sampling to Detach Alignment Modules from Large Language Models [48.15777554876988]
  Traditional alignment methods often require retraining large pretrained models.<n>We propose a novel textitResidual Alignment Model (textitRAM) that formalizes the alignment process as a type of importance sampling.<n>We develop a resampling algorithm with iterative token-level decoding to address the common first-token latency issue in comparable methods.
  arXiv  Detail & Related papers  (2025-05-26T08:53:02Z)
• Solver-Informed RL: Grounding Large Language Models for Authentic Optimization Modeling [3.253908111652627]
  Large Language Models (LLMs) often struggle to generate formally correct and usable models against hallucinations.<n>We present a novel framework that significantly improves the authenticity of LLMs for optimization modeling using Reinforcement Learning with Verifiable Reward.
  arXiv  Detail & Related papers  (2025-05-17T02:32:03Z)
• Large Language Model as Meta-Surrogate for Data-Driven Many-Task Optimization: A Proof-of-Principle Study [11.452011929848844]
  This study proposes a novel meta-surrogate framework to assist many-task optimization.<n>We formulate a unified framework for many-task fitness prediction, by defining a universal model with metadata to fit a group of problems.<n>Our framework supports dual-level knowledge transfer -- at both the surrogate and individual levels -- enhancing optimization efficiency and robustness.
  arXiv  Detail & Related papers  (2025-03-11T11:13:11Z)
• Joint Explainability-Performance Optimization With Surrogate Models for AI-Driven Edge Services [3.8688731303365533]
  This paper explores the balance between the predictive accuracy of complex AI models and their approximation by surrogate ones.<n>We introduce a new algorithm based on multi-objective optimization (MOO) to simultaneously minimize both the complex model's prediction error and the error between its outputs and those of the surrogate.
  arXiv  Detail & Related papers  (2025-03-10T19:04:09Z)
• Merging Models on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging [75.93960998357812]
  Deep model merging represents an emerging research direction that combines multiple fine-tuned models to harness their capabilities across different tasks and domains.<n>Current model merging techniques focus on merging all available models simultaneously, with weight matrices-based methods being the predominant approaches.<n>We propose a training-free projection-based continual merging method that processes models sequentially.
  arXiv  Detail & Related papers  (2025-01-16T13:17:24Z)
• It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimization [16.54335356612006]
  The goal of model merging is to combine multiple models, each excelling in different tasks, into a single model that outperforms any of the individual source models. Existing methods rely heavily on human knowledge or intuition. It's difficult to obtain the great model merging configuration in limited evaluations.
  arXiv  Detail & Related papers  (2024-06-29T16:34:23Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)
• When to Update Your Model: Constrained Model-based Reinforcement Learning [50.74369835934703]
  We propose a novel and general theoretical scheme for a non-decreasing performance guarantee of model-based RL (MBRL) Our follow-up derived bounds reveal the relationship between model shifts and performance improvement. A further example demonstrates that learning models from a dynamically-varying number of explorations benefit the eventual returns.
  arXiv  Detail & Related papers  (2022-10-15T17:57:43Z)
• Automatically Learning Compact Quality-aware Surrogates for Optimization Problems [55.94450542785096]
  Solving optimization problems with unknown parameters requires learning a predictive model to predict the values of the unknown parameters and then solving the problem using these values. Recent work has shown that including the optimization problem as a layer in a complex training model pipeline results in predictions of iteration of unobserved decision making. We show that we can improve solution quality by learning a low-dimensional surrogate model of a large optimization problem.
  arXiv  Detail & Related papers  (2020-06-18T19:11:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.