On Faithfulness and Coherence of Language Explanations for Recommendation Systems

• URL: http://arxiv.org/abs/2209.05409v1
• Date: Mon, 12 Sep 2022 17:00:31 GMT
• Title: On Faithfulness and Coherence of Language Explanations for Recommendation Systems
• Authors: Zhouhang Xie, Julian McAuley and Bodhisattwa Prasad Majumder
• Abstract summary: This work probes state-of-the-art models and their review generation component. We show that the generated explanations are brittle and need further evaluation before being taken as literal rationales for the estimated ratings.
• Score: 8.143715142450876
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reviews contain rich information about product characteristics and user interests and thus are commonly used to boost recommender system performance. Specifically, previous work show that jointly learning to perform review generation improves rating prediction performance. Meanwhile, these model-produced reviews serve as recommendation explanations, providing the user with insights on predicted ratings. However, while existing models could generate fluent, human-like reviews, it is unclear to what degree the reviews fully uncover the rationale behind the jointly predicted rating. In this work, we perform a series of evaluations that probes state-of-the-art models and their review generation component. We show that the generated explanations are brittle and need further evaluation before being taken as literal rationales for the estimated ratings.

Related papers

• Analytical and Empirical Study of Herding Effects in Recommendation Systems [72.6693986712978]
  We study how to manage product ratings via rating aggregation rules and shortlisted representative reviews. We show that proper recency aware rating aggregation rules can improve the speed of convergence in Amazon and TripAdvisor.
  arXiv  Detail & Related papers  (2024-08-20T14:29:23Z)
• Beyond Thumbs Up/Down: Untangling Challenges of Fine-Grained Feedback for Text-to-Image Generation [67.88747330066049]
  Fine-grained feedback captures nuanced distinctions in image quality and prompt-alignment. We show that demonstrating its superiority to coarse-grained feedback is not automatic. We identify key challenges in eliciting and utilizing fine-grained feedback.
  arXiv  Detail & Related papers  (2024-06-24T17:19:34Z)
• CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation [87.44350003888646]
  Eval-Instruct can acquire pointwise grading critiques with pseudo references and revise these critiques via multi-path prompting. CritiqueLLM is empirically shown to outperform ChatGPT and all the open-source baselines.
  arXiv  Detail & Related papers  (2023-11-30T16:52:42Z)
• Explainable Recommender with Geometric Information Bottleneck [25.703872435370585]
  We propose to incorporate a geometric prior learnt from user-item interactions into a variational network. Latent factors from an individual user-item pair can be used for both recommendation and explanation generation. Experimental results on three e-commerce datasets show that our model significantly improves the interpretability of a variational recommender.
  arXiv  Detail & Related papers  (2023-05-09T10:38:36Z)
• Towards Personalized Review Summarization by Modeling Historical Reviews from Customer and Product Separately [59.61932899841944]
  Review summarization is a non-trivial task that aims to summarize the main idea of the product review in the E-commerce website. We propose the Heterogeneous Historical Review aware Review Summarization Model (HHRRS) We employ a multi-task framework that conducts the review sentiment classification and summarization jointly.
  arXiv  Detail & Related papers  (2023-01-27T12:32:55Z)
• Factual and Informative Review Generation for Explainable Recommendation [41.403493319602816]
  Previous models' generated content often contain factual hallucinations. Inspired by recent success in using retrieved content in addition to parametric knowledge for generation, we propose to augment the generator with a personalized retriever. Experiments on Yelp, TripAdvisor, and Amazon Movie Reviews dataset show our model could generate explanations that more reliably entail existing reviews, are more diverse, and are rated more informative by human evaluators.
  arXiv  Detail & Related papers  (2022-09-12T16:46:47Z)
• Dynamic Review-based Recommenders [1.5427245397603195]
  We leverage the known power of reviews to enhance rating predictions in a way that respects the causality of review generation. Our representations are time-interval aware and thus yield a continuous-time representation of the dynamics.
  arXiv  Detail & Related papers  (2021-10-27T20:17:47Z)
• SIFN: A Sentiment-aware Interactive Fusion Network for Review-based Item Recommendation [48.1799451277808]
  We propose a Sentiment-aware Interactive Fusion Network (SIFN) for review-based item recommendation. We first encode user/item reviews via BERT and propose a light-weighted sentiment learner to extract semantic features of each review. Then, we propose a sentiment prediction task that guides the sentiment learner to extract sentiment-aware features via explicit sentiment labels.
  arXiv  Detail & Related papers  (2021-08-18T08:04:38Z)
• Evaluations and Methods for Explanation through Robustness Analysis [117.7235152610957]
  We establish a novel set of evaluation criteria for such feature based explanations by analysis. We obtain new explanations that are loosely necessary and sufficient for a prediction. We extend the explanation to extract the set of features that would move the current prediction to a target class.
  arXiv  Detail & Related papers  (2020-05-31T05:52:05Z)
• How Useful are Reviews for Recommendation? A Critical Review and Potential Improvements [8.471274313213092]
  We investigate a growing body of work that seeks to improve recommender systems through the use of review text. Our initial findings reveal several discrepancies in reported results, partly due to copying results across papers despite changes in experimental settings or data pre-processing. Further investigation calls for discussion on a much larger problem about the "importance" of user reviews for recommendation.
  arXiv  Detail & Related papers  (2020-05-25T16:30:05Z)
• Context-aware Helpfulness Prediction for Online Product Reviews [34.47368084659301]
  We propose a neural deep learning model that predicts the helpfulness score of a review. This model is based on convolutional neural network (CNN) and a context-aware encoding mechanism. We validated our model on human annotated dataset and the result shows that our model significantly outperforms existing models for helpfulness prediction.
  arXiv  Detail & Related papers  (2020-04-27T18:19:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.