User Preferences for Large Language Model versus Template-Based Explanations of Movie Recommendations: A Pilot Study

• URL: http://arxiv.org/abs/2409.06297v1
• Date: Tue, 10 Sep 2024 07:51:53 GMT
• Title: User Preferences for Large Language Model versus Template-Based Explanations of Movie Recommendations: A Pilot Study
• Authors: Julien Albert, Martin Balfroid, Miriam Doh, Jeremie Bogaert, Luca La Fisca, Liesbet De Vos, Bryan Renard, Vincent Stragier, Emmanuel Jean,
• Abstract summary: Large language models (LLMs) can generate more resonant explanations for recommender systems. We conducted a pilot study with 25 participants. Although subject to high variance, preliminary findings suggest that LLM-based explanations may provide a richer and more engaging user experience.
• Score: 0.6965384453064829
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recommender systems have become integral to our digital experiences, from online shopping to streaming platforms. Still, the rationale behind their suggestions often remains opaque to users. While some systems employ a graph-based approach, offering inherent explainability through paths associating recommended items and seed items, non-experts could not easily understand these explanations. A popular alternative is to convert graph-based explanations into textual ones using a template and an algorithm, which we denote here as ''template-based'' explanations. Yet, these can sometimes come across as impersonal or uninspiring. A novel method would be to employ large language models (LLMs) for this purpose, which we denote as ''LLM-based''. To assess the effectiveness of LLMs in generating more resonant explanations, we conducted a pilot study with 25 participants. They were presented with three explanations: (1) traditional template-based, (2) LLM-based rephrasing of the template output, and (3) purely LLM-based explanations derived from the graph-based explanations. Although subject to high variance, preliminary findings suggest that LLM-based explanations may provide a richer and more engaging user experience, further aligning with user expectations. This study sheds light on the potential limitations of current explanation methods and offers promising directions for leveraging large language models to improve user satisfaction and trust in recommender systems.

Related papers

• ReasoningRec: Bridging Personalized Recommendations and Human-Interpretable Explanations through LLM Reasoning [15.049688896236821]
  This paper presents ReasoningRec, a reasoning-based recommendation framework. ReasoningRec bridges the gap between recommendations and human-interpretable explanations. Empirical evaluations demonstrate that ReasoningRec surpasses state-of-the-art methods by up to 12.5% in recommendation prediction.
  arXiv  Detail & Related papers  (2024-10-30T16:37:04Z)
• MAPLE: Enhancing Review Generation with Multi-Aspect Prompt LEarning in Explainable Recommendation [12.874105550787514]
  We propose a personalized, aspect-controlled model called Multi-Aspect Prompt LEarner (MAPLE) Experiments on two real-world review datasets in restaurant domain show that MAPLE outperforms the baseline review-generation models in terms of text.
  arXiv  Detail & Related papers  (2024-08-19T10:12:52Z)
• Show, Don't Tell: Aligning Language Models with Demonstrated Feedback [54.10302745921713]
  Demonstration ITerated Task Optimization (DITTO) directly aligns language model outputs to a user's demonstrated behaviors. We evaluate DITTO's ability to learn fine-grained style and task alignment across domains such as news articles, emails, and blog posts.
  arXiv  Detail & Related papers  (2024-06-02T23:13:56Z)
• Learning to Generate Explainable Stock Predictions using Self-Reflective Large Language Models [54.21695754082441]
  We propose a framework to teach Large Language Models (LLMs) to generate explainable stock predictions. A reflective agent learns how to explain past stock movements through self-reasoning, while the PPO trainer trains the model to generate the most likely explanations. Our framework can outperform both traditional deep-learning and LLM methods in prediction accuracy and Matthews correlation coefficient.
  arXiv  Detail & Related papers  (2024-02-06T03:18:58Z)
• Unlocking the Potential of Large Language Models for Explainable Recommendations [55.29843710657637]
  It remains uncertain what impact replacing the explanation generator with the recently emerging large language models (LLMs) would have. In this study, we propose LLMXRec, a simple yet effective two-stage explainable recommendation framework. By adopting several key fine-tuning techniques, controllable and fluent explanations can be well generated.
  arXiv  Detail & Related papers  (2023-12-25T09:09:54Z)
• RecExplainer: Aligning Large Language Models for Explaining Recommendation Models [50.74181089742969]
  Large language models (LLMs) have demonstrated remarkable intelligence in understanding, reasoning, and instruction following. This paper presents the initial exploration of using LLMs as surrogate models to explain black-box recommender models. To facilitate an effective alignment, we introduce three methods: behavior alignment, intention alignment, and hybrid alignment.
  arXiv  Detail & Related papers  (2023-11-18T03:05:43Z)
• Evaluating and Explaining Large Language Models for Code Using Syntactic Structures [74.93762031957883]
  This paper introduces ASTxplainer, an explainability method specific to Large Language Models for code. At its core, ASTxplainer provides an automated method for aligning token predictions with AST nodes. We perform an empirical evaluation on 12 popular LLMs for code using a curated dataset of the most popular GitHub projects.
  arXiv  Detail & Related papers  (2023-08-07T18:50:57Z)
• Exploring Large Language Model for Graph Data Understanding in Online Job Recommendations [63.19448893196642]
  We present a novel framework that harnesses the rich contextual information and semantic representations provided by large language models to analyze behavior graphs. By leveraging this capability, our framework enables personalized and accurate job recommendations for individual users.
  arXiv  Detail & Related papers  (2023-07-10T11:29:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.