Selective Ensemble Based on Preference-Directed Multi-Objective Bandits

• URL: http://arxiv.org/abs/2606.21929v1
• Date: Sat, 20 Jun 2026 07:52:46 GMT
• Title: Selective Ensemble Based on Preference-Directed Multi-Objective Bandits
• Abstract summary: We formalize a sequential decision problem under partially specified linear preferences.<n>We then propose a preference-directed upper confidence bound (PrefUCB) algorithm.<n> Experiments on large pre-trained model selective ensemble tasks and online asset allocation under institutional mandates validate our method.
• Score: 90.75513823660775
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Selective ensemble for modern machine learning systems requires choosing promising model candidates under limited evaluation budgets, while downstream tasks often specify only partial preferences over capabilities such as accuracy, robustness, and reasoning. This setting naturally gives rise to a sequential decision problem under partially specified linear preferences. We formalize it as preference-directed multi-objective bandits (PDMOB), where admissible trade-offs are represented by a polyhedral preference cone. Based on this formulation, we introduce Pareto $C$-optimality, which recovers standard Pareto optimality and single-weight scalarization as special cases. We then propose the preference-directed upper confidence bound (PrefUCB) algorithm, which maintains directional confidence intervals to guide exploration. We analyze both indicator-based and gap-weighted regret, and establish instance-dependent logarithmic bounds for both criteria, recovering the optimal logarithmic dependence on the horizon $T$ in classical special cases. Experiments on large pre-trained model selective ensemble tasks and online asset allocation under institutional mandates validate the efficacy of our method.

Related papers

• Preference Data Selection for Mitigating the Alignment Tax in Large Language Models [52.18310687523229]
  BALIGN is a balanced data selection strategy that mitigates catastrophic forgetting while optimizing alignment efficacy.<n>By aggregating three key data-centric features into a unified composite risk score, BALIGN systematically filters out high-risk preference samples.<n>Experiments on standard human preference datasets demonstrate that BALIGN strongly preserves foundational capabilities without compromising alignment gains.
  arXiv  Detail & Related papers  (2026-08-25T07:58:42Z)
• Provably Efficient Personalized Multi-Objective Bandits with Proactive Conversational Queries [24.496126767935937]
  We formalize a proactive query-based framework in which user queries provide structured preference signals.<n>We prove that proactive queries accelerate preference estimation and yield improved regret scaling over prior preference-aware MO-MAB methods.
  arXiv  Detail & Related papers  (2026-06-07T02:19:59Z)
• Efficient Online Conformal Selection with Limited Feedback [14.326282143930769]
  We address the problem of conformal selection, where an agent must select a minimal subset of options to ensure that at least one success'' is identified.<n>We show that the simple Adaptive Conformal Inference (ACI) update rule, when applied to the appropriate control parameter or dual variable, is both adversarially valid and efficient.
  arXiv  Detail & Related papers  (2026-05-14T15:22:46Z)
• An Interactive Framework for Finding the Optimal Trade-off in Differential Privacy [20.038766371144526]
  We introduce Differential privacy (DP) as the standard for privacy-preserving analysis, and introduce a fundamental trade-off between privacy guarantees and model performance.<n>In particular, we present the user with hypothetical trade-off curves and ask them to pick their preferred trade-off.<n>Our experiments on differentially private logistic regression and deep transfer learning across six real-world datasets show that our method converges to the optimal privacy-accuracy trade-off.
  arXiv  Detail & Related papers  (2025-09-04T15:02:10Z)
• Multi-Level Aware Preference Learning: Enhancing RLHF for Complex Multi-Instruction Tasks [81.44256822500257]
  RLHF has emerged as a predominant approach for aligning artificial intelligence systems with human preferences.<n> RLHF exhibits insufficient compliance capabilities when confronted with complex multi-instruction tasks.<n>We propose a novel Multi-level Aware Preference Learning (MAPL) framework, capable of enhancing multi-instruction capabilities.
  arXiv  Detail & Related papers  (2025-05-19T08:33:11Z)
• Calibrated Multi-Preference Optimization for Aligning Diffusion Models [90.15024547673785]
  Calibrated Preference Optimization (CaPO) is a novel method to align text-to-image (T2I) diffusion models.<n>CaPO incorporates the general preference from multiple reward models without human annotated data.<n> Experimental results show that CaPO consistently outperforms prior methods.
  arXiv  Detail & Related papers  (2025-02-04T18:59:23Z)
• An incremental preference elicitation-based approach to learning potentially non-monotonic preferences in multi-criteria sorting [53.36437745983783]
  We first construct a max-margin optimization-based model to model potentially non-monotonic preferences. We devise information amount measurement methods and question selection strategies to pinpoint the most informative alternative in each iteration. Two incremental preference elicitation-based algorithms are developed to learn potentially non-monotonic preferences.
  arXiv  Detail & Related papers  (2024-09-04T14:36:20Z)
• Risk-Controlling Model Selection via Guided Bayesian Optimization [35.53469358591976]
  We find a configuration that adheres to user-specified limits on certain risks while being useful with respect to other conflicting metrics. Our method identifies a set of optimal configurations residing in a designated region of interest. We demonstrate the effectiveness of our approach on a range of tasks with multiple desiderata, including low error rates, equitable predictions, handling spurious correlations, managing rate and distortion in generative models, and reducing computational costs.
  arXiv  Detail & Related papers  (2023-12-04T07:29:44Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.