Mixture-Model Preference Learning for Many-Objective Bayesian Optimization

• URL: http://arxiv.org/abs/2603.28410v1
• Date: Mon, 30 Mar 2026 13:18:43 GMT
• Title: Mixture-Model Preference Learning for Many-Objective Bayesian Optimization
• Abstract summary: We propose a Bayesian framework that learns a small set of latent preference archetypes.<n>We design hybrid queries that target information about mode identity and within-mode trade-offs.<n>Our method outperforms standard baselines on synthetic and real-world many-objective benchmarks.
• Score: 21.451177846149417
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Preference-based many-objective optimization faces two obstacles: an expanding space of trade-offs and heterogeneous, context-dependent human value structures. Towards this, we propose a Bayesian framework that learns a small set of latent preference archetypes rather than assuming a single fixed utility function, modelling them as components of a Dirichlet-process mixture with uncertainty over both archetypes and their weights. To query efficiently, we designing hybrid queries that target information about (i) mode identity and (ii) within-mode trade-offs. Under mild assumptions, we provide a simple regret guarantee for the resulting mixture-aware Bayesian optimization procedure. Empirically, our method outperforms standard baselines on synthetic and real-world many-objective benchmarks, and mixture-aware diagnostics reveal structure that regret alone fails to capture.

Related papers

• CoMerge: Conflict-Driven Preference Optimization for Multi-Task Model Merging [59.836482609414816]
  We propose a conflict-driven preference optimization framework for model merging (CoMerge)<n>CoMerge enables the model to mitigate parameter-space conflicts while preserving task-specific capabilities.<n>Experiments show that CoMerge achieves an average normalized performance of 0.9968 on MergeBench.
  arXiv  Detail & Related papers  (2026-09-02T08:21:28Z)
• Enhancing Bayesian Optimization and Active Learning Through Kernel Diversity [13.065036862403966]
  This paper proposes a unified framework, KENDO, that integrates Ensemble Gaussian Processes (EGP) with disagreement-aware acquisition strategies.<n>KENDO-BO achieves competitive or superior optimization performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-08-25T15:35:47Z)
• Unification and Optimization of Robust Supervised Learning [24.343315694527476]
  We propose a tractable training procedure that decomposes robust learning into sequential stages.<n>This results in a unified design space that can compose and configure strategies suited to the task at hand.
  arXiv  Detail & Related papers  (2026-05-27T08:47:50Z)
• Bayesian Optimization for Molecules Should Be Pareto-Aware [6.877358955271652]
  Multi-objective Bayesian optimization (MOBO) provides a principled framework for navigating trade-offs in molecular design.<n>We benchmark a simple MOBO strategy against a simple fixed-weight scalarized baseline using Expected Improvement (EI)<n>Our results show that even strong deterministic instantiations can underperform in low-data regimes.
  arXiv  Detail & Related papers  (2025-07-18T07:12:19Z)
• Discrete Markov Bridge [93.64996843697278]
  We propose a novel framework specifically designed for discrete representation learning, called Discrete Markov Bridge.<n>Our approach is built upon two key components: Matrix Learning and Score Learning.
  arXiv  Detail & Related papers  (2025-05-26T09:32:12Z)
• Multi-Level Aware Preference Learning: Enhancing RLHF for Complex Multi-Instruction Tasks [81.44256822500257]
  RLHF has emerged as a predominant approach for aligning artificial intelligence systems with human preferences.<n> RLHF exhibits insufficient compliance capabilities when confronted with complex multi-instruction tasks.<n>We propose a novel Multi-level Aware Preference Learning (MAPL) framework, capable of enhancing multi-instruction capabilities.
  arXiv  Detail & Related papers  (2025-05-19T08:33:11Z)
• An incremental preference elicitation-based approach to learning potentially non-monotonic preferences in multi-criteria sorting [53.36437745983783]
  We first construct a max-margin optimization-based model to model potentially non-monotonic preferences. We devise information amount measurement methods and question selection strategies to pinpoint the most informative alternative in each iteration. Two incremental preference elicitation-based algorithms are developed to learn potentially non-monotonic preferences.
  arXiv  Detail & Related papers  (2024-09-04T14:36:20Z)
• Diffusion Model for Data-Driven Black-Box Optimization [54.25693582870226]
  We focus on diffusion models, a powerful generative AI technology, and investigate their potential for black-box optimization. We study two practical types of labels: 1) noisy measurements of a real-valued reward function and 2) human preference based on pairwise comparisons. Our proposed method reformulates the design optimization problem into a conditional sampling problem, which allows us to leverage the power of diffusion models.
  arXiv  Detail & Related papers  (2024-03-20T00:41:12Z)
• Pseudo-Bayesian Optimization [7.556071491014536]
  We study an axiomatic framework that elicits the minimal requirements to guarantee black-box optimization convergence.<n>We show how using simple local regression, and a suitable "randomized prior" construction to quantify uncertainty, not only guarantees convergence but also consistently outperforms state-of-the-art benchmarks.
  arXiv  Detail & Related papers  (2023-10-15T07:55:28Z)
• Differentiable Multi-Target Causal Bayesian Experimental Design [43.76697029708785]
  We introduce a gradient-based approach for the problem of Bayesian optimal experimental design to learn causal models in a batch setting. Existing methods rely on greedy approximations to construct a batch of experiments. We propose a conceptually simple end-to-end gradient-based optimization procedure to acquire a set of optimal intervention target-state pairs.
  arXiv  Detail & Related papers  (2023-02-21T11:32:59Z)
• Tree ensemble kernels for Bayesian optimization with known constraints over mixed-feature spaces [54.58348769621782]
  Tree ensembles can be well-suited for black-box optimization tasks such as algorithm tuning and neural architecture search. Two well-known challenges in using tree ensembles for black-box optimization are (i) effectively quantifying model uncertainty for exploration and (ii) optimizing over the piece-wise constant acquisition function. Our framework performs as well as state-of-the-art methods for unconstrained black-box optimization over continuous/discrete features and outperforms competing methods for problems combining mixed-variable feature spaces and known input constraints.
  arXiv  Detail & Related papers  (2022-07-02T16:59:37Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.