Choosing the Best of Both Worlds: Diverse and Novel Recommendations
through Multi-Objective Reinforcement Learning
- URL: http://arxiv.org/abs/2110.15097v1
- Date: Thu, 28 Oct 2021 13:22:45 GMT
- Title: Choosing the Best of Both Worlds: Diverse and Novel Recommendations
through Multi-Objective Reinforcement Learning
- Authors: Dusan Stamenkovic, Alexandros Karatzoglou, Ioannis Arapakis, Xin Xin,
Kleomenis Katevas
- Abstract summary: We introduce Scalarized Multi-Objective Reinforcement Learning (SMORL) for the Recommender Systems (RS) setting.
SMORL agent augments standard recommendation models with additional RL layers that enforce it to simultaneously satisfy three principal objectives: accuracy, diversity, and novelty of recommendations.
Our experimental results on two real-world datasets reveal a substantial increase in aggregate diversity, a moderate increase in accuracy, reduced repetitiveness of recommendations, and demonstrate the importance of reinforcing diversity and novelty as complementary objectives.
- Score: 68.45370492516531
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Since the inception of Recommender Systems (RS), the accuracy of the
recommendations in terms of relevance has been the golden criterion for
evaluating the quality of RS algorithms. However, by focusing on item
relevance, one pays a significant price in terms of other important metrics:
users get stuck in a "filter bubble" and their array of options is
significantly reduced, hence degrading the quality of the user experience and
leading to churn. Recommendation, and in particular session-based/sequential
recommendation, is a complex task with multiple - and often conflicting
objectives - that existing state-of-the-art approaches fail to address.
In this work, we take on the aforementioned challenge and introduce
Scalarized Multi-Objective Reinforcement Learning (SMORL) for the RS setting, a
novel Reinforcement Learning (RL) framework that can effectively address
multi-objective recommendation tasks. The proposed SMORL agent augments
standard recommendation models with additional RL layers that enforce it to
simultaneously satisfy three principal objectives: accuracy, diversity, and
novelty of recommendations. We integrate this framework with four
state-of-the-art session-based recommendation models and compare it with a
single-objective RL agent that only focuses on accuracy. Our experimental
results on two real-world datasets reveal a substantial increase in aggregate
diversity, a moderate increase in accuracy, reduced repetitiveness of
recommendations, and demonstrate the importance of reinforcing diversity and
novelty as complementary objectives.
Related papers
- A Thorough Performance Benchmarking on Lightweight Embedding-based Recommender Systems [67.52782366565658]
State-of-the-art recommender systems (RSs) depend on categorical features, which ecoded by embedding vectors, resulting in excessively large embedding tables.
Despite the prosperity of lightweight embedding-based RSs, a wide diversity is seen in evaluation protocols.
This study investigates various LERS' performance, efficiency, and cross-task transferability via a thorough benchmarking process.
arXiv Detail & Related papers (2024-06-25T07:45:00Z) - LIRE: listwise reward enhancement for preference alignment [27.50204023448716]
We propose a gradient-based reward optimization approach that incorporates the offline rewards of multiple responses into a streamlined listwise framework.
LIRE is straightforward to implement, requiring minimal parameter tuning, and seamlessly aligns with the pairwise paradigm.
Our experiments demonstrate that LIRE consistently outperforms existing methods across several benchmarks on dialogue and summarization tasks.
arXiv Detail & Related papers (2024-05-22T10:21:50Z) - MORL-Prompt: An Empirical Analysis of Multi-Objective Reinforcement
Learning for Discrete Prompt Optimization [49.60729578316884]
RL-based techniques can be used to search for prompts that maximize a set of user-specified reward functions.
Current techniques focus on maximizing the average of reward functions, which does not necessarily lead to prompts that achieve balance across rewards.
In this paper, we adapt several techniques for multi-objective optimization to RL-based discrete prompt optimization.
arXiv Detail & Related papers (2024-02-18T21:25:09Z) - D2RLIR : an improved and diversified ranking function in interactive
recommendation systems based on deep reinforcement learning [0.3058685580689604]
This paper proposes a deep reinforcement learning based recommendation system by utilizing Actor-Critic architecture.
The proposed model is able to generate a diverse while relevance recommendation list based on the user's preferences.
arXiv Detail & Related papers (2021-10-28T13:11:29Z) - Information Directed Reward Learning for Reinforcement Learning [64.33774245655401]
We learn a model of the reward function that allows standard RL algorithms to achieve high expected return with as few expert queries as possible.
In contrast to prior active reward learning methods designed for specific types of queries, IDRL naturally accommodates different query types.
We support our findings with extensive evaluations in multiple environments and with different types of queries.
arXiv Detail & Related papers (2021-02-24T18:46:42Z) - RRCN: A Reinforced Random Convolutional Network based Reciprocal
Recommendation Approach for Online Dating [26.033983596934338]
We propose a novel reinforced random convolutional network (RRCN) approach for the reciprocal recommendation task.
We evaluate the proposed RRCN against a number of both baselines and the state-of-the-art approaches on two real-world datasets.
arXiv Detail & Related papers (2020-11-25T08:55:17Z) - Sample-Rank: Weak Multi-Objective Recommendations Using Rejection
Sampling [0.5156484100374059]
We introduce a method involving multi-goal sampling followed by ranking for user-relevance (Sample-Rank) to nudge recommendations towards multi-objective goals of the marketplace.
The proposed method's novelty is that it reduces the MO recommendation problem to sampling from a desired multi-goal distribution then using it to build a production-friendly learning-to-rank model.
arXiv Detail & Related papers (2020-08-24T09:17:18Z) - Self-Supervised Reinforcement Learning for Recommender Systems [77.38665506495553]
We propose self-supervised reinforcement learning for sequential recommendation tasks.
Our approach augments standard recommendation models with two output layers: one for self-supervised learning and the other for RL.
Based on such an approach, we propose two frameworks namely Self-Supervised Q-learning(SQN) and Self-Supervised Actor-Critic(SAC)
arXiv Detail & Related papers (2020-06-10T11:18:57Z) - Sequential Recommendation with Self-Attentive Multi-Adversarial Network [101.25533520688654]
We present a Multi-Factor Generative Adversarial Network (MFGAN) for explicitly modeling the effect of context information on sequential recommendation.
Our framework is flexible to incorporate multiple kinds of factor information, and is able to trace how each factor contributes to the recommendation decision over time.
arXiv Detail & Related papers (2020-05-21T12:28:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.