QUIVER: Cost-Aware Adaptive Preference Querying in Surrogate-Assisted Evolutionary Multi-Objective Optimization

• URL: http://arxiv.org/abs/2605.04267v1
• Date: Tue, 05 May 2026 20:02:35 GMT
• Title: QUIVER: Cost-Aware Adaptive Preference Querying in Surrogate-Assisted Evolutionary Multi-Objective Optimization
• Abstract summary: We introduce QUIVER, a surrogate-assisted evolutionary multi-objective that adaptively chooses between objective evaluations and preference queries.<n>We show QUIVER achieves the lowest final utility regret on challenging WFG problems.<n>This adaptive modality selection demonstrates cost-aware preference learning in action.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Interactive multi-objective optimization systems face a budget allocation dilemma: one can spend resources on expensive objective evaluations or on eliciting decision-maker preferences that identify the relevant region of the Pareto set. Moreover, preference elicitation itself spans modalities with different information content and cognitive burden, ranging from cheap, noisy pairwise preference statements (PS) to richer but costlier indifference adjustments (IA). We study cost-aware optimization under an unknown scalarization and introduce QUIVER (Query-Informed Value Estimation for Regret), a surrogate-assisted evolutionary multi-objective optimizer that adaptively chooses between objective evaluations and heterogeneous preference queries. At each step, QUIVER selects the next action by maximizing the expected decision-quality improvement per unit total cost. Across DTLZ and WFG benchmarks under synthetic decision-maker models, QUIVER achieves the lowest final utility regret on challenging WFG problems (utility regret of 2.14 on WFG4, 2.82 on WFG9: a 25% improvement over baselines), outperforming all single-modality baselines. We analyze how the optimal mix of PS and IA adapts to problem difficulty: on easy problems (DTLZ2), QUIVER selects 80\% PS queries; on hard problems (WFG9), it shifts to 35% IA queries. This adaptive modality selection demonstrates cost-aware preference learning in action.

Related papers

• Diversity-Driven Offline Multi-Objective Optimization via Nested Pareto Set Learning [11.1141573484419]
  Multi-objective optimization (MOO) has emerged as a powerful approach to solving complex optimization problems involving multiple objectives.<n>This paper proposes Diversity-driven Offline Multi-Objective Optimization (DOMOO), which aims to find out a diverse and high-quality set of solutions.<n>Experiments on synthetic and real-world benchmarks show that DOMOO achieves the best average rank across tasks in both convergence and diversity.
  arXiv  Detail & Related papers  (2026-06-13T05:09:25Z)
• MO-CAPO: Multi-Objective Cost-Aware Prompt Optimization [3.2307940915114473]
  Large language models (LLMs) achieve strong performance across a wide range of tasks but are highly sensitive to prompt design.<n>Existing methods predominantly focus on performance alone, ignoring competing objectives such as inference cost or latency.<n>We introduce MO-CAPO, a novel multi-objective prompt optimization algorithm that jointly optimize performance and inference cost.
  arXiv  Detail & Related papers  (2026-05-15T14:56:27Z)
• GIFT: Global Irreplaceability Frame Targeting for Efficient Video Understanding [63.62445065977448]
  GIFT: Global Irreplaceability Frame Targeting is a training-free framework that selects frames by assessing their intrinsic irreplaceability.<n>We show that GIFT achieves a maximum average improvement of 12.5% across long-form video benchmarks on LLaVA-Video-7B compared to uniform sampling.
  arXiv  Detail & Related papers  (2026-03-26T06:21:41Z)
• ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference [60.958331943869126]
  ODAR-Expert is an adaptive routing framework that optimize the accuracy-efficiency trade-off via principled resource allocation.<n>We show strong and consistent gains, including 98.2% accuracy on MATH and 54.8% on Humanity's Last Exam.
  arXiv  Detail & Related papers  (2026-02-27T05:22:01Z)
• Parametric Expensive Multi-Objective Optimization via Generative Solution Modeling [34.344228998247225]
  This paper introduces the first parametric multi-objective Bayesian that learns this inverse model by alternating between acquisition-driven search and generative models.<n>We theoretically justify the faster convergence by leveraging inter-task synergies through task-aware Gaussian processes.
  arXiv  Detail & Related papers  (2025-11-12T15:13:27Z)
• Parametric Pareto Set Learning for Expensive Multi-Objective Optimization [16.780031024741223]
  Parametric multi-objective optimization (PMO) addresses the challenge of solving an infinite family of multi-objective optimization problems.<n>Traditional methods require re-execution for each parameter configuration, leading to prohibitive costs when objective evaluations are computationally expensive.
  arXiv  Detail & Related papers  (2025-11-08T03:05:28Z)
• Reverse Preference Optimization for Complex Instruction Following [61.39734201711077]
  We propose a simple yet effective method called Reverse Preference Optimization (RPO)<n>It mitigates noise in preference pairs by dynamically reversing the constraints within the instruction to ensure the chosen response is perfect.<n>RPO scales effectively across model sizes, with the 70B RPO model surpassing GPT-4o.
  arXiv  Detail & Related papers  (2025-05-28T09:44:27Z)
• Regularized infill criteria for multi-objective Bayesian optimization with application to aircraft design [0.565395466029518]
  The proposed method in this paper extends the super efficient global optimization with mixture of experts SEGOMOE to solve constrained multiobjective problems.<n>The merit of the proposed approaches are shown on known multiobjective benchmark problems with and without constraints.<n>Preliminary results show a reduction of the total cost in terms of function evaluations by a factor of 20 compared to the evolutionary algorithm NSGA-II.
  arXiv  Detail & Related papers  (2025-04-11T16:24:40Z)
• Self-Steering Optimization: Autonomous Preference Optimization for Large Language Models [79.84205827056907]
  We present Self-Steering Optimization ($SSO$), an algorithm that autonomously generates high-quality preference data.<n>$SSO$ employs a specialized optimization objective to build a data generator from the policy model itself, which is used to produce accurate and on-policy data.<n>Our evaluation shows that $SSO$ consistently outperforms baselines in human preference alignment and reward optimization.
  arXiv  Detail & Related papers  (2024-10-22T16:04:03Z)
• Training Greedy Policy for Proposal Batch Selection in Expensive Multi-Objective Combinatorial Optimization [52.80408805368928]
  We introduce a novel greedy-style subset selection algorithm for batch acquisition. Our experiments on the red fluorescent proteins show that our proposed method achieves the baseline performance in 1.69x fewer queries.
  arXiv  Detail & Related papers  (2024-06-21T05:57:08Z)
• Adaptive Preference Scaling for Reinforcement Learning with Human Feedback [103.36048042664768]
  Reinforcement learning from human feedback (RLHF) is a prevalent approach to align AI systems with human values. We propose a novel adaptive preference loss, underpinned by distributionally robust optimization (DRO) Our method is versatile and can be readily adapted to various preference optimization frameworks.
  arXiv  Detail & Related papers  (2024-06-04T20:33:22Z)
• Multi-Objective Bayesian Optimization with Active Preference Learning [18.066263838953223]
  We propose a Bayesian optimization (BO) approach to identifying the most preferred solution in a multi-objective optimization (MOO) problem. To minimize the interaction cost with the decision maker (DM), we also propose an active learning strategy for the preference estimation.
  arXiv  Detail & Related papers  (2023-11-22T15:24:36Z)
• Studying Evolutionary Solution Adaption Using a Flexibility Benchmark Based on a Metal Cutting Process [39.05320053926048]
  We consider optimizing for different production requirements from the viewpoint of a bio-inspired framework for system flexibility.<n>We study the flexibility of NSGA-II, which we extend by two variants: 1) varying goals, which optimize solutions for two tasks simultaneously to obtain in-between source solutions expected to be more adaptable, and 2) active-inactive genotype, which accommodates different possibilities that can be activated or deactivated.
  arXiv  Detail & Related papers  (2023-05-31T12:07:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.