MoSH: Modeling Multi-Objective Tradeoffs with Soft and Hard Bounds

• URL: http://arxiv.org/abs/2412.06154v1
• Date: Mon, 09 Dec 2024 02:32:20 GMT
• Title: MoSH: Modeling Multi-Objective Tradeoffs with Soft and Hard Bounds
• Authors: Edward Chen, Natalie Dullerud, Thomas Niedermayr, Elizabeth Kidd, Ransalu Senanayake, Pang Wei Koh, Sanmi Koyejo, Carlos Guestrin,
• Abstract summary: We introduce a novel conceptual framework that operationalizes soft-hard functions, SHFs, which allow for the DM to intuitively impose soft and hard bounds on each objective.<n>We show that many practical problems fit within the SHF framework and provide extensive empirical validation on diverse domains.<n>Specifically, for brachytherapy, our approach returns a compact set of points with over 3% greater SHF-defined utility than the next best approach.
• Score: 29.347695311801864
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Countless science and engineering applications in multi-objective optimization (MOO) necessitate that decision-makers (DMs) select a Pareto-optimal solution which aligns with their preferences. Evaluating individual solutions is often expensive, necessitating cost-sensitive optimization techniques. Due to competing objectives, the space of trade-offs is also expansive -- thus, examining the full Pareto frontier may prove overwhelming to a DM. Such real-world settings generally have loosely-defined and context-specific desirable regions for each objective function that can aid in constraining the search over the Pareto frontier. We introduce a novel conceptual framework that operationalizes these priors using soft-hard functions, SHFs, which allow for the DM to intuitively impose soft and hard bounds on each objective -- which has been lacking in previous MOO frameworks. Leveraging a novel minimax formulation for Pareto frontier sampling, we propose a two-step process for obtaining a compact set of Pareto-optimal points which respect the user-defined soft and hard bounds: (1) densely sample the Pareto frontier using Bayesian optimization, and (2) sparsify the selected set to surface to the user, using robust submodular function optimization. We prove that (2) obtains the optimal compact Pareto-optimal set of points from (1). We further show that many practical problems fit within the SHF framework and provide extensive empirical validation on diverse domains, including brachytherapy, engineering design, and large language model personalization. Specifically, for brachytherapy, our approach returns a compact set of points with over 3% greater SHF-defined utility than the next best approach. Among the other diverse experiments, our approach consistently leads in utility, allowing the DM to reach >99% of their maximum possible desired utility within validation of 5 points.

Related papers

• Preference-Guided Diffusion for Multi-Objective Offline Optimization [64.08326521234228]
  We propose a preference-guided diffusion model for offline multi-objective optimization. Our guidance is a preference model trained to predict the probability that one design dominates another. Our results highlight the effectiveness of classifier-guided diffusion models in generating diverse and high-quality solutions.
  arXiv  Detail & Related papers  (2025-03-21T16:49:38Z)
• Self-Improvement Towards Pareto Optimality: Mitigating Preference Conflicts in Multi-Objective Alignment [74.25832963097658]
  Multi-Objective Alignment (MOA) aims to align responses with multiple human preference objectives. We find that DPO-based MOA approaches suffer from widespread preference conflicts in the data.
  arXiv  Detail & Related papers  (2025-02-20T08:27:00Z)
• Provably Efficient Multi-Objective Bandit Algorithms under Preference-Centric Customization [24.533662423325943]
  We study a preference-aware MO-MAB framework in the presence of explicit user preference. This is the first theoretical study of customized MO-MAB optimization with explicit user preferences.
  arXiv  Detail & Related papers  (2025-02-19T06:06:13Z)
• Improving Pareto Set Learning for Expensive Multi-objective Optimization via Stein Variational Hypernetworks [4.124390946636935]
  Expensive multi-objective optimization problems (EMOPs) are common in real-world scenarios where evaluating objective functions is costly. We propose a novel approach called SVH-PSL, which integrates Stein Variational Gradient Descent (SVGD) with Hypernetworks. Our method addresses the issues of fragmented surrogate models and pseudo-local optima by collectively moving particles in a manner that smooths out the solution space.
  arXiv  Detail & Related papers  (2024-12-23T06:05:45Z)
• C-MORL: Multi-Objective Reinforcement Learning through Efficient Discovery of Pareto Front [9.04360155372014]
  Constrained MORL is a seamless bridge between constrained policy optimization and MORL. Our algorithm achieves more consistent and superior performances in terms of hypervolume, expected utility, and sparsity on both discrete and continuous control tasks.
  arXiv  Detail & Related papers  (2024-10-03T06:13:56Z)
• Preference-Optimized Pareto Set Learning for Blackbox Optimization [1.9628841617148691]
  No single solution exists that can optimize all the objectives simultaneously. In a typical MOO problem, the goal is to find a set of optimum solutions (Pareto set) that trades off the preferences among objectives. Our formulation leads to a bilevel optimization problem that can be solved by e.g. differentiable cross-entropy methods.
  arXiv  Detail & Related papers  (2024-08-19T13:23:07Z)
• Decoding-Time Language Model Alignment with Multiple Objectives [116.42095026960598]
  Existing methods primarily focus on optimizing LMs for a single reward function, limiting their adaptability to varied objectives. Here, we propose $textbfmulti-objective decoding (MOD)$, a decoding-time algorithm that outputs the next token from a linear combination of predictions. We show why existing approaches can be sub-optimal even in natural settings and obtain optimality guarantees for our method.
  arXiv  Detail & Related papers  (2024-06-27T02:46:30Z)
• Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion [53.33473557562837]
  Solving multi-objective optimization problems for large deep neural networks is a challenging task due to the complexity of the loss landscape and the expensive computational cost. We propose a practical and scalable approach to solve this problem via mixture of experts (MoE) based model fusion. By ensembling the weights of specialized single-task models, the MoE module can effectively capture the trade-offs between multiple objectives.
  arXiv  Detail & Related papers  (2024-06-14T07:16:18Z)
• Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment [103.12563033438715]
  Alignment in artificial intelligence pursues consistency between model responses and human preferences as well as values. Existing alignment techniques are mostly unidirectional, leading to suboptimal trade-offs and poor flexibility over various objectives. We introduce controllable preference optimization (CPO), which explicitly specifies preference scores for different objectives.
  arXiv  Detail & Related papers  (2024-02-29T12:12:30Z)
• Multi-Objective Bayesian Optimization with Active Preference Learning [18.066263838953223]
  We propose a Bayesian optimization (BO) approach to identifying the most preferred solution in a multi-objective optimization (MOO) problem. To minimize the interaction cost with the decision maker (DM), we also propose an active learning strategy for the preference estimation.
  arXiv  Detail & Related papers  (2023-11-22T15:24:36Z)
• Optimizing Solution-Samplers for Combinatorial Problems: The Landscape of Policy-Gradient Methods [52.0617030129699]
  We introduce a novel theoretical framework for analyzing the effectiveness of DeepMatching Networks and Reinforcement Learning methods. Our main contribution holds for a broad class of problems including Max-and Min-Cut, Max-$k$-Bipartite-Bi, Maximum-Weight-Bipartite-Bi, and Traveling Salesman Problem. As a byproduct of our analysis we introduce a novel regularization process over vanilla descent and provide theoretical and experimental evidence that it helps address vanishing-gradient issues and escape bad stationary points.
  arXiv  Detail & Related papers  (2023-10-08T23:39:38Z)
• BOtied: Multi-objective Bayesian optimization with tied multivariate ranks [33.414682601242006]
  In this paper, we show a natural connection between non-dominated solutions and the extreme quantile of the joint cumulative distribution function. Motivated by this link, we propose the Pareto-compliant CDF indicator and the associated acquisition function, BOtied. Our experiments on a variety of synthetic and real-world problems demonstrate that BOtied outperforms state-of-the-art MOBO acquisition functions.
  arXiv  Detail & Related papers  (2023-06-01T04:50:06Z)
• Maximize to Explore: One Objective Function Fusing Estimation, Planning, and Exploration [87.53543137162488]
  We propose an easy-to-implement online reinforcement learning (online RL) framework called textttMEX. textttMEX integrates estimation and planning components while balancing exploration exploitation automatically. It can outperform baselines by a stable margin in various MuJoCo environments with sparse rewards.
  arXiv  Detail & Related papers  (2023-05-29T17:25:26Z)
• Pareto Set Learning for Expensive Multi-Objective Optimization [5.419608513284392]
  Expensive multi-objective optimization problems can be found in many real-world applications. This paper develops a novel learning-based method to approximate the whole Pareto set for MOBO.
  arXiv  Detail & Related papers  (2022-10-16T09:41:54Z)
• Exact Pareto Optimal Search for Multi-Task Learning and Multi-Criteria Decision-Making [10.914300987810128]
  We show that EPO Search converges to an EPO solution at a linear rate of convergence. We develop new algorithms: PESA-EPO for approximating the PF in a posteriori MCDM, and GP-EPO for elicitation in interactive MCDM. EPO Search scales linearly with the number of variables, which enables its use for deep e-commerce networks.
  arXiv  Detail & Related papers  (2021-08-02T02:13:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.