Reward Constrained Interactive Recommendation with Natural Language Feedback

• URL: http://arxiv.org/abs/2005.01618v1
• Date: Mon, 4 May 2020 16:23:34 GMT
• Title: Reward Constrained Interactive Recommendation with Natural Language Feedback
• Authors: Ruiyi Zhang, Tong Yu, Yilin Shen, Hongxia Jin, Changyou Chen, Lawrence Carin
• Abstract summary: We propose a novel constraint-augmented reinforcement learning (RL) framework to efficiently incorporate user preferences over time. Specifically, we leverage a discriminator to detect recommendations violating user historical preference. Our proposed framework is general and is further extended to the task of constrained text generation.
• Score: 158.8095688415973
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-based interactive recommendation provides richer user feedback and has demonstrated advantages over traditional interactive recommender systems. However, recommendations can easily violate preferences of users from their past natural-language feedback, since the recommender needs to explore new items for further improvement. To alleviate this issue, we propose a novel constraint-augmented reinforcement learning (RL) framework to efficiently incorporate user preferences over time. Specifically, we leverage a discriminator to detect recommendations violating user historical preference, which is incorporated into the standard RL objective of maximizing expected cumulative future rewards. Our proposed framework is general and is further extended to the task of constrained text generation. Empirical results show that the proposed method yields consistent improvement relative to standard RL methods.

Related papers

• Interactive Visualization Recommendation with Hier-SUCB [52.11209329270573]
  We propose an interactive personalized visualization recommendation (PVisRec) system that learns on user feedback from previous interactions. For more interactive and accurate recommendations, we propose Hier-SUCB, a contextual semi-bandit in the PVisRec setting.
  arXiv  Detail & Related papers  (2025-02-05T17:14:45Z)
• Reason4Rec: Large Language Models for Recommendation with Deliberative User Preference Alignment [69.11529841118671]
  We propose a new Deliberative Recommendation task, which incorporates explicit reasoning about user preferences as an additional alignment goal. We then introduce the Reasoning-powered Recommender framework for deliberative user preference alignment.
  arXiv  Detail & Related papers  (2025-02-04T07:17:54Z)
• Preference Discerning with LLM-Enhanced Generative Retrieval [28.309905847867178]
  We propose a new paradigm, which we term preference discerning. In preference dscerning, we explicitly condition a generative sequential recommendation system on user preferences within its context. We generate user preferences using Large Language Models (LLMs) based on user reviews and item-specific data.
  arXiv  Detail & Related papers  (2024-12-11T18:26:55Z)
• RLVF: Learning from Verbal Feedback without Overgeneralization [94.19501420241188]
  We study the problem of incorporating verbal feedback without such overgeneralization. We develop a new method Contextualized Critiques with Constrained Preference Optimization (C3PO) Our approach effectively applies verbal feedback to relevant scenarios while preserving existing behaviors for other contexts.
  arXiv  Detail & Related papers  (2024-02-16T18:50:24Z)
• Hierarchical Reinforcement Learning for Modeling User Novelty-Seeking Intent in Recommender Systems [26.519571240032967]
  We propose a novel hierarchical reinforcement learning-based method to model the hierarchical user novelty-seeking intent. We further incorporate diversity and novelty-related measurement in the reward function of the hierarchical RL (HRL) agent to encourage user exploration.
  arXiv  Detail & Related papers  (2023-06-02T12:02:23Z)
• Editable User Profiles for Controllable Text Recommendation [66.00743968792275]
  We propose LACE, a novel concept value bottleneck model for controllable text recommendations. LACE represents each user with a succinct set of human-readable concepts. It learns personalized representations of the concepts based on user documents.
  arXiv  Detail & Related papers  (2023-04-09T14:52:18Z)
• Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System [11.404192885921498]
  Chat-Rec is a new paradigm for building conversational recommender systems. Chat-Rec is effective in learning user preferences and establishing connections between users and products. In experiments, Chat-Rec effectively improve the results of top-k recommendations and performs better in zero-shot rating prediction task.
  arXiv  Detail & Related papers  (2023-03-25T17:37:43Z)
• Two-Stage Neural Contextual Bandits for Personalised News Recommendation [50.3750507789989]
  Existing personalised news recommendation methods focus on exploiting user interests and ignores exploration in recommendation. We build on contextual bandits recommendation strategies which naturally address the exploitation-exploration trade-off. We use deep learning representations for users and news, and generalise the neural upper confidence bound (UCB) policies to generalised additive UCB and bilinear UCB.
  arXiv  Detail & Related papers  (2022-06-26T12:07:56Z)
• Offline Meta-level Model-based Reinforcement Learning Approach for Cold-Start Recommendation [27.17948754183511]
  Reinforcement learning has shown great promise in optimizing long-term user interest in recommender systems. Existing RL-based recommendation methods need a large number of interactions for each user to learn a robust recommendation policy. We propose a meta-level model-based reinforcement learning approach for fast user adaptation.
  arXiv  Detail & Related papers  (2020-12-04T08:58:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.