On the Opportunities and Challenges of Offline Reinforcement Learning for Recommender Systems

• URL: http://arxiv.org/abs/2308.11336v1
• Date: Tue, 22 Aug 2023 10:28:02 GMT
• Title: On the Opportunities and Challenges of Offline Reinforcement Learning for Recommender Systems
• Authors: Xiaocong Chen, Siyu Wang, Julian McAuley, Dietmar Jannach and Lina Yao
• Abstract summary: Reinforcement learning serves as potent tool for modeling dynamic user interests within recommender systems. Recent strides in offline reinforcement learning present a new perspective. Despite being a burgeoning field, works centered on recommender systems utilizing offline reinforcement learning remain limited.
• Score: 36.608400817940236
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning serves as a potent tool for modeling dynamic user interests within recommender systems, garnering increasing research attention of late. However, a significant drawback persists: its poor data efficiency, stemming from its interactive nature. The training of reinforcement learning-based recommender systems demands expensive online interactions to amass adequate trajectories, essential for agents to learn user preferences. This inefficiency renders reinforcement learning-based recommender systems a formidable undertaking, necessitating the exploration of potential solutions. Recent strides in offline reinforcement learning present a new perspective. Offline reinforcement learning empowers agents to glean insights from offline datasets and deploy learned policies in online settings. Given that recommender systems possess extensive offline datasets, the framework of offline reinforcement learning aligns seamlessly. Despite being a burgeoning field, works centered on recommender systems utilizing offline reinforcement learning remain limited. This survey aims to introduce and delve into offline reinforcement learning within recommender systems, offering an inclusive review of existing literature in this domain. Furthermore, we strive to underscore prevalent challenges, opportunities, and future pathways, poised to propel research in this evolving field.

Related papers

• Small Dataset, Big Gains: Enhancing Reinforcement Learning by Offline Pre-Training with Model Based Augmentation [59.899714450049494]
  offline pre-training can produce sub-optimal policies and lead to degraded online reinforcement learning performance. We propose a model-based data augmentation strategy to maximize the benefits of offline reinforcement learning pre-training and reduce the scale of data needed to be effective.
  arXiv  Detail & Related papers  (2023-12-15T14:49:41Z)
• Embedding in Recommender Systems: A Survey [67.67966158305603]
  A crucial aspect is embedding techniques that covert the high-dimensional discrete features, such as user and item IDs, into low-dimensional continuous vectors. Applying embedding techniques captures complex entity relationships and has spurred substantial research. This survey covers embedding methods like collaborative filtering, self-supervised learning, and graph-based techniques.
  arXiv  Detail & Related papers  (2023-10-28T06:31:06Z)
• Online Matching: A Real-time Bandit System for Large-scale Recommendations [23.954049092470548]
  Online Matching is a scalable closed-loop bandit system learning from users' direct feedback on items in real time. Diag-LinUCB is a novel extension of the LinUCB algorithm to enable distributed updates of bandits parameter in a scalable and timely manner.
  arXiv  Detail & Related papers  (2023-07-29T05:46:27Z)
• Interactive Search Based on Deep Reinforcement Learning [4.353144350714567]
  The project mainly establishes a virtual user environment for offline training. At the same time, we tried to improve a reinforcement learning algorithm based on bi-clustering to expand the action space and recommended path space of the recommendation agent.
  arXiv  Detail & Related papers  (2020-12-09T15:23:53Z)
• Generative Inverse Deep Reinforcement Learning for Online Recommendation [62.09946317831129]
  We propose a novel inverse reinforcement learning approach, namely InvRec, for online recommendation. InvRec extracts the reward function from user's behaviors automatically, for online recommendation.
  arXiv  Detail & Related papers  (2020-11-04T12:12:25Z)
• Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems [108.81683598693539]
  offline reinforcement learning algorithms hold tremendous promise for making it possible to turn large datasets into powerful decision making engines. We will aim to provide the reader with an understanding of these challenges, particularly in the context of modern deep reinforcement learning methods.
  arXiv  Detail & Related papers  (2020-05-04T17:00:15Z)
• Knowledge-guided Deep Reinforcement Learning for Interactive Recommendation [49.32287384774351]
  Interactive recommendation aims to learn from dynamic interactions between items and users to achieve responsiveness and accuracy. We propose Knowledge-Guided deep Reinforcement learning to harness the advantages of both reinforcement learning and knowledge graphs for interactive recommendation.
  arXiv  Detail & Related papers  (2020-04-17T05:26:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.