Large Language Model-Enhanced Reinforcement Learning for Diverse and Novel Recommendations

• URL: http://arxiv.org/abs/2507.21274v1
• Date: Mon, 28 Jul 2025 19:00:40 GMT
• Title: Large Language Model-Enhanced Reinforcement Learning for Diverse and Novel Recommendations
• Authors: Jiin Woo, Alireza Bagheri Garakani, Tianchen Zhou, Zhishen Huang, Yan Gao,
• Abstract summary: We propose LAAC (LLM-guided Adversarial Actor Critic), a novel method that leverages large language models to suggest novel items.<n>We show that LAAC outperforms existing baselines in diversity, novelty, and accuracy, while remaining robust on imbalanced data.
• Score: 6.949170757786365
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recommendation systems, diversity and novelty are essential for capturing varied user preferences and encouraging exploration, yet many systems prioritize click relevance. While reinforcement learning (RL) has been explored to improve diversity, it often depends on random exploration that may not align with user interests. We propose LAAC (LLM-guided Adversarial Actor Critic), a novel method that leverages large language models (LLMs) as reference policies to suggest novel items, while training a lightweight policy to refine these suggestions using system-specific data. The method formulates training as a bilevel optimization between actor and critic networks, enabling the critic to selectively favor promising novel actions and the actor to improve its policy beyond LLM recommendations. To mitigate overestimation of unreliable LLM suggestions, we apply regularization that anchors critic values for unexplored items close to well-estimated dataset actions. Experiments on real-world datasets show that LAAC outperforms existing baselines in diversity, novelty, and accuracy, while remaining robust on imbalanced data, effectively integrating LLM knowledge without expensive fine-tuning.

Related papers

• Evaluating Position Bias in Large Language Model Recommendations [3.430780143519032]
  Large Language Models (LLMs) are being increasingly explored as general-purpose tools for recommendation tasks.<n>We show that LLM-based recommendation models suffer from position bias, where the order of candidate items in a prompt can disproportionately influence the recommendations produced by LLMs.<n>We introduce a new prompting strategy to mitigate the position bias of LLM recommendation models called Ranking via Iterative SElection.
  arXiv  Detail & Related papers  (2025-08-04T03:30:26Z)
• DeepRec: Towards a Deep Dive Into the Item Space with Large Language Model Based Recommendation [83.21140655248624]
  Large language models (LLMs) have been introduced into recommender systems (RSs)<n>We propose DeepRec, a novel LLM-based RS that enables autonomous multi-turn interactions between LLMs and TRMs for deep exploration of the item space.<n> Experiments on public datasets demonstrate that DeepRec significantly outperforms both traditional and LLM-based baselines.
  arXiv  Detail & Related papers  (2025-05-22T15:49:38Z)
• User Feedback Alignment for LLM-powered Exploration in Large-scale Recommendation Systems [26.652050105571206]
  Exploration, the act of broadening user experiences beyond their established preferences, is challenging in large-scale recommendation systems.<n>This paper introduces a novel approach that combines hierarchical planning with LLM inference-time scaling.<n>We show significant gains in both user satisfaction (measured by watch activity and active user counts) and exploration diversity.
  arXiv  Detail & Related papers  (2025-04-07T21:44:12Z)
• Large Language Model driven Policy Exploration for Recommender Systems [50.70228564385797]
  offline RL policies trained on static user data are vulnerable to distribution shift when deployed in dynamic online environments.<n>Online RL-based RS also face challenges in production deployment due to the risks of exposing users to untrained or unstable policies.<n>Large Language Models (LLMs) offer a promising solution to mimic user objectives and preferences for pre-training policies offline.<n>We propose an Interaction-Augmented Learned Policy (iALP) that utilizes user preferences distilled from an LLM.
  arXiv  Detail & Related papers  (2025-01-23T16:37:44Z)
• Real-Time Personalization for LLM-based Recommendation with Customized In-Context Learning [57.28766250993726]
  This work explores adapting to dynamic user interests without any model updates. Existing Large Language Model (LLM)-based recommenders often lose the in-context learning ability during recommendation tuning. We propose RecICL, which customizes recommendation-specific in-context learning for real-time recommendations.
  arXiv  Detail & Related papers  (2024-10-30T15:48:36Z)
• MMREC: LLM Based Multi-Modal Recommender System [2.3113916776957635]
  This paper presents a novel approach to enhancing recommender systems by leveraging Large Language Models (LLMs) and deep learning techniques.<n>The proposed framework aims to improve the accuracy and relevance of recommendations by incorporating multi-modal information processing and by the use of unified latent space representation.
  arXiv  Detail & Related papers  (2024-08-08T04:31:29Z)
• How Can LLM Guide RL? A Value-Based Approach [68.55316627400683]
  Reinforcement learning (RL) has become the de facto standard practice for sequential decision-making problems by improving future acting policies with feedback. Recent developments in large language models (LLMs) have showcased impressive capabilities in language understanding and generation, yet they fall short in exploration and self-improvement capabilities. We develop an algorithm named LINVIT that incorporates LLM guidance as a regularization factor in value-based RL, leading to significant reductions in the amount of data needed for learning.
  arXiv  Detail & Related papers  (2024-02-25T20:07:13Z)
• Exploring the Impact of Large Language Models on Recommender Systems: An Extensive Review [2.780460221321639]
  The paper underscores the significance of Large Language Models in reshaping recommender systems. LLMs exhibit exceptional proficiency in recommending items, showcasing their adeptness in comprehending intricacies of language. Despite their transformative potential, challenges persist, including sensitivity to input prompts, occasional misinterpretations, and unforeseen recommendations.
  arXiv  Detail & Related papers  (2024-02-11T00:24:17Z)
• A Survey on Large Language Models for Recommendation [77.91673633328148]
  Large Language Models (LLMs) have emerged as powerful tools in the field of Natural Language Processing (NLP) This survey presents a taxonomy that categorizes these models into two major paradigms, respectively Discriminative LLM for Recommendation (DLLM4Rec) and Generative LLM for Recommendation (GLLM4Rec)
  arXiv  Detail & Related papers  (2023-05-31T13:51:26Z)
• Choosing the Best of Both Worlds: Diverse and Novel Recommendations through Multi-Objective Reinforcement Learning [68.45370492516531]
  We introduce Scalarized Multi-Objective Reinforcement Learning (SMORL) for the Recommender Systems (RS) setting. SMORL agent augments standard recommendation models with additional RL layers that enforce it to simultaneously satisfy three principal objectives: accuracy, diversity, and novelty of recommendations. Our experimental results on two real-world datasets reveal a substantial increase in aggregate diversity, a moderate increase in accuracy, reduced repetitiveness of recommendations, and demonstrate the importance of reinforcing diversity and novelty as complementary objectives.
  arXiv  Detail & Related papers  (2021-10-28T13:22:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.