Generative Explore-Exploit: Training-free Optimization of Generative Recommender Systems using LLM Optimizers

• URL: http://arxiv.org/abs/2406.05255v1
• Date: Fri, 7 Jun 2024 20:41:59 GMT
• Title: Generative Explore-Exploit: Training-free Optimization of Generative Recommender Systems using LLM Optimizers
• Authors: Lütfi Kerem Senel, Besnik Fetahu, Davis Yoshida, Zhiyu Chen, Giuseppe Castellucci, Nikhita Vedula, Jason Choi, Shervin Malmasi,
• Abstract summary: We present a training-free approach for optimizing generative recommenders. We propose a generative explore-exploit method that can not only exploit generated items with high engagement, but also actively explore and discover hidden population preferences.
• Score: 29.739736497044664
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recommender systems are widely used to suggest engaging content, and Large Language Models (LLMs) have given rise to generative recommenders. Such systems can directly generate items, including for open-set tasks like question suggestion. While the world knowledge of LLMs enable good recommendations, improving the generated content through user feedback is challenging as continuously fine-tuning LLMs is prohibitively expensive. We present a training-free approach for optimizing generative recommenders by connecting user feedback loops to LLM-based optimizers. We propose a generative explore-exploit method that can not only exploit generated items with known high engagement, but also actively explore and discover hidden population preferences to improve recommendation quality. We evaluate our approach on question generation in two domains (e-commerce and general knowledge), and model user feedback with Click Through Rate (CTR). Experiments show our LLM-based explore-exploit approach can iteratively improve recommendations, and consistently increase CTR. Ablation analysis shows that generative exploration is key to learning user preferences, avoiding the pitfalls of greedy exploit-only approaches. A human evaluation strongly supports our quantitative findings.

Related papers

• LANE: Logic Alignment of Non-tuning Large Language Models and Online Recommendation Systems for Explainable Reason Generation [15.972926854420619]
  Leveraging large language models (LLMs) offers new opportunities for comprehensive recommendation logic generation. Fine-tuning LLM models for recommendation tasks incurs high computational costs and alignment issues with existing systems. In this work, our proposed effective strategy LANE aligns LLMs with online recommendation systems without additional LLMs tuning.
  arXiv  Detail & Related papers  (2024-07-03T06:20:31Z)
• LLM-Powered Explanations: Unraveling Recommendations Through Subgraph Reasoning [40.53821858897774]
  We introduce a novel recommender that synergies Large Language Models (LLMs) and Knowledge Graphs (KGs) to enhance the recommendation and provide interpretable results. Our approach significantly enhances both the effectiveness and interpretability of recommender systems.
  arXiv  Detail & Related papers  (2024-06-22T14:14:03Z)
• Aligning Large Language Models for Controllable Recommendations [32.82292871427888]
  We introduce a collection of supervised learning tasks, augmented with labels derived from a conventional recommender model. We then develop a reinforcement learning-based alignment procedure to strengthen LLMs' aptitude in responding to users' intentions. Our method markedly advances the capability of LLMs to comply with instructions within recommender systems, while sustaining a high level of accuracy performance.
  arXiv  Detail & Related papers  (2024-03-08T05:23:27Z)
• Unlocking the Potential of Large Language Models for Explainable Recommendations [55.29843710657637]
  It remains uncertain what impact replacing the explanation generator with the recently emerging large language models (LLMs) would have. In this study, we propose LLMXRec, a simple yet effective two-stage explainable recommendation framework. By adopting several key fine-tuning techniques, controllable and fluent explanations can be well generated.
  arXiv  Detail & Related papers  (2023-12-25T09:09:54Z)
• Empowering Few-Shot Recommender Systems with Large Language Models -- Enhanced Representations [0.0]
  Large language models (LLMs) offer novel insights into tackling the few-shot scenarios encountered by explicit feedback-based recommender systems. Our study can inspire researchers to delve deeper into the multifaceted dimensions of LLMs's involvement in recommender systems.
  arXiv  Detail & Related papers  (2023-12-21T03:50:09Z)
• LLMRec: Benchmarking Large Language Models on Recommendation Task [54.48899723591296]
  The application of Large Language Models (LLMs) in the recommendation domain has not been thoroughly investigated. We benchmark several popular off-the-shelf LLMs on five recommendation tasks, including rating prediction, sequential recommendation, direct recommendation, explanation generation, and review summarization. The benchmark results indicate that LLMs displayed only moderate proficiency in accuracy-based tasks such as sequential and direct recommendation.
  arXiv  Detail & Related papers  (2023-08-23T16:32:54Z)
• Recommender Systems in the Era of Large Language Models (LLMs) [62.0129013439038]
  Large Language Models (LLMs) have revolutionized the fields of Natural Language Processing (NLP) and Artificial Intelligence (AI) We conduct a comprehensive review of LLM-empowered recommender systems from various aspects including Pre-training, Fine-tuning, and Prompting.
  arXiv  Detail & Related papers  (2023-07-05T06:03:40Z)
• GenRec: Large Language Model for Generative Recommendation [41.22833600362077]
  This paper presents an innovative approach to recommendation systems using large language models (LLMs) based on text data. GenRec uses LLM's understanding ability to interpret context, learn user preferences, and generate relevant recommendation. Our research underscores the potential of LLM-based generative recommendation in revolutionizing the domain of recommendation systems.
  arXiv  Detail & Related papers  (2023-07-02T02:37:07Z)
• A Survey on Large Language Models for Recommendation [77.91673633328148]
  Large Language Models (LLMs) have emerged as powerful tools in the field of Natural Language Processing (NLP) This survey presents a taxonomy that categorizes these models into two major paradigms, respectively Discriminative LLM for Recommendation (DLLM4Rec) and Generative LLM for Recommendation (GLLM4Rec)
  arXiv  Detail & Related papers  (2023-05-31T13:51:26Z)
• Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach [83.62750225073341]
  We consider recommendation as instruction following by large language models (LLMs) We first design a general instruction format for describing the preference, intention, task form and context of a user in natural language. Then we manually design 39 instruction templates and automatically generate a large amount of user-personalized instruction data.
  arXiv  Detail & Related papers  (2023-05-11T17:39:07Z)
• Reward Constrained Interactive Recommendation with Natural Language Feedback [158.8095688415973]
  We propose a novel constraint-augmented reinforcement learning (RL) framework to efficiently incorporate user preferences over time. Specifically, we leverage a discriminator to detect recommendations violating user historical preference. Our proposed framework is general and is further extended to the task of constrained text generation.
  arXiv  Detail & Related papers  (2020-05-04T16:23:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.