Disentangling Likes and Dislikes in Personalized Generative Explainable Recommendation

• URL: http://arxiv.org/abs/2410.13248v1
• Date: Thu, 17 Oct 2024 06:15:00 GMT
• Title: Disentangling Likes and Dislikes in Personalized Generative Explainable Recommendation
• Authors: Ryotaro Shimizu, Takashi Wada, Yu Wang, Johannes Kruse, Sean O'Brien, Sai HtaungKham, Linxin Song, Yuya Yoshikawa, Yuki Saito, Fugee Tsung, Masayuki Goto, Julian McAuley,
• Abstract summary: We introduce new datasets and evaluation methods that focus on the users' sentiments. We construct the datasets by explicitly extracting users' positive and negative opinions from their post-purchase reviews. We propose to evaluate systems based on whether the generated explanations align well with the users' sentiments.
• Score: 26.214148426964794
• License:
• Abstract: Recent research on explainable recommendation generally frames the task as a standard text generation problem, and evaluates models simply based on the textual similarity between the predicted and ground-truth explanations. However, this approach fails to consider one crucial aspect of the systems: whether their outputs accurately reflect the users' (post-purchase) sentiments, i.e., whether and why they would like and/or dislike the recommended items. To shed light on this issue, we introduce new datasets and evaluation methods that focus on the users' sentiments. Specifically, we construct the datasets by explicitly extracting users' positive and negative opinions from their post-purchase reviews using an LLM, and propose to evaluate systems based on whether the generated explanations 1) align well with the users' sentiments, and 2) accurately identify both positive and negative opinions of users on the target items. We benchmark several recent models on our datasets and demonstrate that achieving strong performance on existing metrics does not ensure that the generated explanations align well with the users' sentiments. Lastly, we find that existing models can provide more sentiment-aware explanations when the users' (predicted) ratings for the target items are directly fed into the models as input. We will release our code and datasets upon acceptance.

Related papers

• Contextualized Evaluations: Taking the Guesswork Out of Language Model Evaluations [85.81295563405433]
  Language model users often issue queries that lack specification, where the context under which a query was issued is not explicit. We present contextualized evaluations, a protocol that synthetically constructs context surrounding an under-specified query and provides it during evaluation. We find that the presence of context can 1) alter conclusions drawn from evaluation, even flipping win rates between model pairs, 2) nudge evaluators to make fewer judgments based on surface-level criteria, like style, and 3) provide new insights about model behavior across diverse contexts.
  arXiv  Detail & Related papers  (2024-11-11T18:58:38Z)
• What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception [53.4840989321394]
  We analyze the effect of rationales generated by QA models to support their answers. We present users with incorrect answers and corresponding rationales in various formats. We measure the effectiveness of this feedback in patching these rationales through in-context learning.
  arXiv  Detail & Related papers  (2023-11-16T04:26:32Z)
• Topology-aware Debiased Self-supervised Graph Learning for Recommendation [6.893289671937124]
  We propose Topology-aware De Self-supervised Graph Learning ( TDSGL) for recommendation. TDSGL constructs contrastive pairs according to the semantic similarity between users (items) Our results show that the proposed model outperforms the state-of-the-art models significantly on three public datasets.
  arXiv  Detail & Related papers  (2023-10-24T14:16:19Z)
• Exploiting Rich Textual User-Product Context for Improving Sentiment Analysis [21.840121866597563]
  We propose a method to explicitly employ historical reviews belonging to the same user/product to initialize representations. Experiments on IMDb, Yelp-2013 and Yelp-2014 benchmarks show that our approach substantially outperforms previous state-of-the-art.
  arXiv  Detail & Related papers  (2022-12-17T14:57:52Z)
• Graph-based Extractive Explainer for Recommendations [38.278148661173525]
  We develop a graph attentive neural network model that seamlessly integrates user, item, attributes, and sentences for extraction-based explanation. To balance individual sentence relevance, overall attribute coverage, and content redundancy, we solve an integer linear programming problem to make the final selection of sentences.
  arXiv  Detail & Related papers  (2022-02-20T04:56:10Z)
• SIFN: A Sentiment-aware Interactive Fusion Network for Review-based Item Recommendation [48.1799451277808]
  We propose a Sentiment-aware Interactive Fusion Network (SIFN) for review-based item recommendation. We first encode user/item reviews via BERT and propose a light-weighted sentiment learner to extract semantic features of each review. Then, we propose a sentiment prediction task that guides the sentiment learner to extract sentiment-aware features via explicit sentiment labels.
  arXiv  Detail & Related papers  (2021-08-18T08:04:38Z)
• Set2setRank: Collaborative Set to Set Ranking for Implicit Feedback based Recommendation [59.183016033308014]
  In this paper, we explore the unique characteristics of the implicit feedback and propose Set2setRank framework for recommendation. Our proposed framework is model-agnostic and can be easily applied to most recommendation prediction approaches.
  arXiv  Detail & Related papers  (2021-05-16T08:06:22Z)
• E-commerce Query-based Generation based on User Review [1.484852576248587]
  We propose a novel seq2seq based text generation model to generate answers to user's question based on reviews posted by previous users. Given a user question and/or target sentiment polarity, we extract aspects of interest and generate an answer that summarizes previous relevant user reviews.
  arXiv  Detail & Related papers  (2020-11-11T04:58:31Z)
• A Unified Dual-view Model for Review Summarization and Sentiment Classification with Inconsistency Loss [51.448615489097236]
  Acquiring accurate summarization and sentiment from user reviews is an essential component of modern e-commerce platforms. We propose a novel dual-view model that jointly improves the performance of these two tasks. Experiment results on four real-world datasets from different domains demonstrate the effectiveness of our model.
  arXiv  Detail & Related papers  (2020-06-02T13:34:11Z)
• Evaluating Models' Local Decision Boundaries via Contrast Sets [119.38387782979474]
  We propose a new annotation paradigm for NLP that helps to close systematic gaps in the test data. We demonstrate the efficacy of contrast sets by creating them for 10 diverse NLP datasets. Although our contrast sets are not explicitly adversarial, model performance is significantly lower on them than on the original test sets.
  arXiv  Detail & Related papers  (2020-04-06T14:47:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.