Deep Pareto Reinforcement Learning for Multi-Objective Recommender Systems

• URL: http://arxiv.org/abs/2407.03580v2
• Date: Tue, 09 Jul 2024 18:54:59 GMT
• Title: Deep Pareto Reinforcement Learning for Multi-Objective Recommender Systems
• Authors: Pan Li, Alexander Tuzhilin,
• Abstract summary: optimizing multiple objectives simultaneously is an important task for recommendation platforms. Existing multi-objective recommender systems do not systematically consider such dynamic relationships.
• Score: 60.91599969408029
• License:
• Abstract: Optimizing multiple objectives simultaneously is an important task for recommendation platforms to improve their performance. However, this task is particularly challenging since the relationships between different objectives are heterogeneous across different consumers and dynamically fluctuating according to different contexts. Especially in those cases when objectives become conflicting with each other, the result of recommendations will form a pareto-frontier, where the improvements of any objective comes at the cost of a performance decrease of another objective. Existing multi-objective recommender systems do not systematically consider such dynamic relationships; instead, they balance between these objectives in a static and uniform manner, resulting in only suboptimal multi-objective recommendation performance. In this paper, we propose a Deep Pareto Reinforcement Learning (DeepPRL) approach, where we (1) comprehensively model the complex relationships between multiple objectives in recommendations; (2) effectively capture personalized and contextual consumer preference for each objective to provide better recommendations; (3) optimize both the short-term and the long-term performance of multi-objective recommendations. As a result, our method achieves significant pareto-dominance over the state-of-the-art baselines in the offline experiments. Furthermore, we conducted a controlled experiment at the video streaming platform of Alibaba, where our method simultaneously improved three conflicting business objectives over the latest production system significantly, demonstrating its tangible economic impact in practice.

Related papers

• Learning Recommender Systems with Soft Target: A Decoupled Perspective [49.83787742587449]
  We propose a novel decoupled soft label optimization framework to consider the objectives as two aspects by leveraging soft labels. We present a sensible soft-label generation algorithm that models a label propagation algorithm to explore users' latent interests in unobserved feedback via neighbors.
  arXiv  Detail & Related papers  (2024-10-09T04:20:15Z)
• Reduced-Rank Multi-objective Policy Learning and Optimization [57.978477569678844]
  In practice, causal researchers do not have a single outcome in mind a priori. In government-assisted social benefit programs, policymakers collect many outcomes to understand the multidimensional nature of poverty. We present a data-driven dimensionality-reduction methodology for multiple outcomes in the context of optimal policy learning.
  arXiv  Detail & Related papers  (2024-04-29T08:16:30Z)
• Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment [103.12563033438715]
  Alignment in artificial intelligence pursues consistency between model responses and human preferences as well as values. Existing alignment techniques are mostly unidirectional, leading to suboptimal trade-offs and poor flexibility over various objectives. We introduce controllable preference optimization (CPO), which explicitly specifies preference scores for different objectives.
  arXiv  Detail & Related papers  (2024-02-29T12:12:30Z)
• Dynamic value alignment through preference aggregation of multiple objectives [0.0]
  We present a methodology for dynamic value alignment, where the values that are to be aligned with are dynamically changing. We apply this approach to extend Deep $Q$-Learning to accommodate multiple objectives and evaluate this method on a simplified two-leg intersection.
  arXiv  Detail & Related papers  (2023-10-09T17:07:26Z)
• Alleviating Search Bias in Bayesian Evolutionary Optimization with Many Heterogeneous Objectives [9.139734850798124]
  We deal with multi-objective optimization problems with heterogeneous objectives (HE-MOPs) A new acquisition function that mitigates search bias towards the fast objectives is suggested. We demonstrate the effectiveness of the proposed algorithm by testing it on widely used multi-/many-objective benchmark problems.
  arXiv  Detail & Related papers  (2022-08-25T17:07:40Z)
• Leveraging Trust for Joint Multi-Objective and Multi-Fidelity Optimization [0.0]
  This paper investigates a novel approach to Bayesian multi-objective and multi-fidelity (MOMF) optimization. We suggest the innovative use of a trust metric to support simultaneous optimization of multiple objectives and data sources. Our methods offer broad applicability in solving simulation problems in fields such as plasma physics and fluid dynamics.
  arXiv  Detail & Related papers  (2021-12-27T20:55:26Z)
• Choosing the Best of Both Worlds: Diverse and Novel Recommendations through Multi-Objective Reinforcement Learning [68.45370492516531]
  We introduce Scalarized Multi-Objective Reinforcement Learning (SMORL) for the Recommender Systems (RS) setting. SMORL agent augments standard recommendation models with additional RL layers that enforce it to simultaneously satisfy three principal objectives: accuracy, diversity, and novelty of recommendations. Our experimental results on two real-world datasets reveal a substantial increase in aggregate diversity, a moderate increase in accuracy, reduced repetitiveness of recommendations, and demonstrate the importance of reinforcing diversity and novelty as complementary objectives.
  arXiv  Detail & Related papers  (2021-10-28T13:22:45Z)
• Optimizing Interactive Systems via Data-Driven Objectives [70.3578528542663]
  We propose an approach that infers the objective directly from observed user interactions. These inferences can be made regardless of prior knowledge and across different types of user behavior. We introduce Interactive System (ISO), a novel algorithm that uses these inferred objectives for optimization.
  arXiv  Detail & Related papers  (2020-06-19T20:49:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.