Dynamic Detection of Relevant Objectives and Adaptation to Preference Drifts in Interactive Evolutionary Multi-Objective Optimization

• URL: http://arxiv.org/abs/2411.04547v1
• Date: Thu, 07 Nov 2024 09:09:06 GMT
• Title: Dynamic Detection of Relevant Objectives and Adaptation to Preference Drifts in Interactive Evolutionary Multi-Objective Optimization
• Authors: Seyed Mahdi Shavarani, Mahmoud Golabi, Richard Allmendinger, Lhassane Idoumghar,
• Abstract summary: We study the dynamic nature of DM preferences, which can evolve throughout the decision-making process and affect the relevance of objectives. We propose methods to discard outdated or conflicting preferences when such shifts occur. Our experimental results demonstrate that the proposed methods effectively manage evolving preferences and significantly enhance the quality and desirability of the solutions produced by the algorithm.
• Score: 2.4374097382908477
• License:
• Abstract: Evolutionary Multi-Objective Optimization Algorithms (EMOAs) are widely employed to tackle problems with multiple conflicting objectives. Recent research indicates that not all objectives are equally important to the decision-maker (DM). In the context of interactive EMOAs, preference information elicited from the DM during the optimization process can be leveraged to identify and discard irrelevant objectives, a crucial step when objective evaluations are computationally expensive. However, much of the existing literature fails to account for the dynamic nature of DM preferences, which can evolve throughout the decision-making process and affect the relevance of objectives. This study addresses this limitation by simulating dynamic shifts in DM preferences within a ranking-based interactive algorithm. Additionally, we propose methods to discard outdated or conflicting preferences when such shifts occur. Building on prior research, we also introduce a mechanism to safeguard relevant objectives that may become trapped in local or global optima due to the diminished correlation with the DM-provided rankings. Our experimental results demonstrate that the proposed methods effectively manage evolving preferences and significantly enhance the quality and desirability of the solutions produced by the algorithm.

Related papers

• Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization [75.1240295759264]
  We propose an effective framework for Bridging and Modeling Correlations in pairwise data, named BMC. We increase the consistency and informativeness of the pairwise preference signals through targeted modifications. We identify that DPO alone is insufficient to model these correlations and capture nuanced variations.
  arXiv  Detail & Related papers  (2024-08-14T11:29:47Z)
• Reduced-Rank Multi-objective Policy Learning and Optimization [57.978477569678844]
  In practice, causal researchers do not have a single outcome in mind a priori. In government-assisted social benefit programs, policymakers collect many outcomes to understand the multidimensional nature of poverty. We present a data-driven dimensionality-reduction methodology for multiple outcomes in the context of optimal policy learning.
  arXiv  Detail & Related papers  (2024-04-29T08:16:30Z)
• Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment [103.12563033438715]
  Alignment in artificial intelligence pursues consistency between model responses and human preferences as well as values. Existing alignment techniques are mostly unidirectional, leading to suboptimal trade-offs and poor flexibility over various objectives. We introduce controllable preference optimization (CPO), which explicitly specifies preference scores for different objectives.
  arXiv  Detail & Related papers  (2024-02-29T12:12:30Z)
• Multi-Objective Bayesian Optimization with Active Preference Learning [18.066263838953223]
  We propose a Bayesian optimization (BO) approach to identifying the most preferred solution in a multi-objective optimization (MOO) problem. To minimize the interaction cost with the decision maker (DM), we also propose an active learning strategy for the preference estimation.
  arXiv  Detail & Related papers  (2023-11-22T15:24:36Z)
• Revisiting the Domain Shift and Sample Uncertainty in Multi-source Active Domain Transfer [69.82229895838577]
  Active Domain Adaptation (ADA) aims to maximally boost model adaptation in a new target domain by actively selecting a limited number of target data to annotate. This setting neglects the more practical scenario where training data are collected from multiple sources. This motivates us to target a new and challenging setting of knowledge transfer that extends ADA from a single source domain to multiple source domains.
  arXiv  Detail & Related papers  (2023-11-21T13:12:21Z)
• Knowledge Transfer for Dynamic Multi-objective Optimization with a Changing Number of Objectives [4.490459770205671]
  We show that the state-of-the-art transfer algorithm for DMOPs with a changing number of objectives lacks sufficient diversity. We propose a knowledge transfer dynamic multi-objective evolutionary algorithm (KTDMOEA) to enhance population diversity after changes.
  arXiv  Detail & Related papers  (2023-06-19T01:54:44Z)
• Cross-Domain Policy Adaptation via Value-Guided Data Filtering [57.62692881606099]
  Generalizing policies across different domains with dynamics mismatch poses a significant challenge in reinforcement learning. We present the Value-Guided Data Filtering (VGDF) algorithm, which selectively shares transitions from the source domain based on the proximity of paired value targets.
  arXiv  Detail & Related papers  (2023-05-28T04:08:40Z)
• Inferring Preferences from Demonstrations in Multi-objective Reinforcement Learning: A Dynamic Weight-based Approach [0.0]
  In multi-objective decision-making, preference inference is the process of inferring the preferences of a decision-maker for different objectives. This research proposes a Dynamic Weight-based Preference Inference algorithm that can infer the preferences of agents acting in multi-objective decision-making problems.
  arXiv  Detail & Related papers  (2023-04-27T11:55:07Z)
• Interactive Evolutionary Multi-Objective Optimization via Learning-to-Rank [8.421614560290609]
  This paper develops a framework for designing preference-based EMO algorithms to find solution(s) of interest (SOI) in an interactive manner. Its core idea is to involve human in the loop of EMO. After every several iterations, the DM is invited to elicit her feedback with regard to a couple of incumbent candidates. By collecting such information, her preference is progressively learned by a learning-to-rank neural network and then applied to guide the baseline EMO algorithm.
  arXiv  Detail & Related papers  (2022-04-06T06:34:05Z)
• Pareto Domain Adaptation [35.48609986914723]
  Domain adaptation (DA) attempts to transfer the knowledge from a labeled source domain to an unlabeled target domain. We propose a new approach to control the overall optimization direction, aiming to cooperatively optimize all training objectives.
  arXiv  Detail & Related papers  (2021-12-08T06:31:13Z)
• MetaAlign: Coordinating Domain Alignment and Classification for Unsupervised Domain Adaptation [84.90801699807426]
  This paper proposes an effective meta-optimization based strategy dubbed MetaAlign. We treat the domain alignment objective and the classification objective as the meta-train and meta-test tasks in a meta-learning scheme. Experimental results demonstrate the effectiveness of our proposed method on top of various alignment-based baseline approaches.
  arXiv  Detail & Related papers  (2021-03-25T03:16:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.