Measuring "Why" in Recommender Systems: a Comprehensive Survey on the Evaluation of Explainable Recommendation

• URL: http://arxiv.org/abs/2202.06466v1
• Date: Mon, 14 Feb 2022 02:58:55 GMT
• Title: Measuring "Why" in Recommender Systems: a Comprehensive Survey on the Evaluation of Explainable Recommendation
• Authors: Xu Chen and Yongfeng Zhang and Ji-Rong Wen
• Abstract summary: This survey is based on more than 100 papers from top-tier conferences like IJCAI, AAAI, TheWebConf, Recsys, UMAP, and IUI.
• Score: 87.82664566721917
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Explainable recommendation has shown its great advantages for improving recommendation persuasiveness, user satisfaction, system transparency, among others. A fundamental problem of explainable recommendation is how to evaluate the explanations. In the past few years, various evaluation strategies have been proposed. However, they are scattered in different papers, and there lacks a systematic and detailed comparison between them. To bridge this gap, in this paper, we comprehensively review the previous work, and provide different taxonomies for them according to the evaluation perspectives and evaluation methods. Beyond summarizing the previous work, we also analyze the (dis)advantages of existing evaluation methods and provide a series of guidelines on how to select them. The contents of this survey are based on more than 100 papers from top-tier conferences like IJCAI, AAAI, TheWebConf, Recsys, UMAP, and IUI, and their complete summarization are presented at https://shimo.im/sheets/VKrpYTcwVH6KXgdy/MODOC/. With this survey, we finally aim to provide a clear and comprehensive review on the evaluation of explainable recommendation.

Related papers

• Revisiting Reciprocal Recommender Systems: Metrics, Formulation, and Method [60.364834418531366]
  We propose five new evaluation metrics that comprehensively and accurately assess the performance of RRS. We formulate the RRS from a causal perspective, formulating recommendations as bilateral interventions. We introduce a reranking strategy to maximize matching outcomes, as measured by the proposed metrics.
  arXiv  Detail & Related papers  (2024-08-19T07:21:02Z)
• Review of Explainable Graph-Based Recommender Systems [2.1711205684359247]
  This review paper discusses state-of-the-art approaches of explainable graph-based recommender systems. It categorizes them based on three aspects: learning methods, explaining methods, and explanation types.
  arXiv  Detail & Related papers  (2024-07-31T21:30:36Z)
• Review-based Recommender Systems: A Survey of Approaches, Challenges and Future Perspectives [11.835903510784735]
  Review-based recommender systems have emerged as a significant sub-field in this domain. We present a categorization of these systems and summarize the state-of-the-art methods, analyzing their unique features, effectiveness, and limitations. We propose potential directions for future research, including the integration of multimodal data, multi-criteria rating information, and ethical considerations.
  arXiv  Detail & Related papers  (2024-05-09T05:45:18Z)
• Impression-Aware Recommender Systems [57.38537491535016]
  Novel data sources bring new opportunities to improve the quality of recommender systems. Researchers may use impressions to refine user preferences and overcome the current limitations in recommender systems research. We present a systematic literature review on recommender systems using impressions.
  arXiv  Detail & Related papers  (2023-08-15T16:16:02Z)
• No Agreement Without Loss: Learning and Social Choice in Peer Review [0.0]
  It may be assumed that each reviewer has her own mapping from the set of features to a recommendation. This introduces an element of arbitrariness known as commensuration bias. Noothigattu, Shah and Procaccia proposed to aggregate reviewer's mapping by minimizing certain loss functions.
  arXiv  Detail & Related papers  (2022-11-03T21:03:23Z)
• Evaluating the Predictive Performance of Positive-Unlabelled Classifiers: a brief critical review and practical recommendations for improvement [77.34726150561087]
  Positive-Unlabelled (PU) learning is a growing area of machine learning. This paper critically reviews the main PU learning evaluation approaches and the choice of predictive accuracy measures in 51 articles proposing PU classifiers.
  arXiv  Detail & Related papers  (2022-06-06T08:31:49Z)
• A Comprehensive Overview of Recommender System and Sentiment Analysis [1.370633147306388]
  This paper gives a comprehensive overview to help researchers who aim to work with recommender system and sentiment analysis. It includes a background of the recommender system concept, including phases, approaches, and performance metrics used in recommender systems. Then, it discusses the sentiment analysis concept and highlights the main points in the sentiment analysis, including level, approaches, and focuses on aspect-based sentiment analysis.
  arXiv  Detail & Related papers  (2021-09-18T01:08:41Z)
• Unsupervised Reference-Free Summary Quality Evaluation via Contrastive Learning [66.30909748400023]
  We propose to evaluate the summary qualities without reference summaries by unsupervised contrastive learning. Specifically, we design a new metric which covers both linguistic qualities and semantic informativeness based on BERT. Experiments on Newsroom and CNN/Daily Mail demonstrate that our new evaluation method outperforms other metrics even without reference summaries.
  arXiv  Detail & Related papers  (2020-10-05T05:04:14Z)
• Fairness-Aware Explainable Recommendation over Knowledge Graphs [73.81994676695346]
  We analyze different groups of users according to their level of activity, and find that bias exists in recommendation performance between different groups. We show that inactive users may be more susceptible to receiving unsatisfactory recommendations, due to insufficient training data for the inactive users. We propose a fairness constrained approach via re-ranking to mitigate this problem in the context of explainable recommendation over knowledge graphs.
  arXiv  Detail & Related papers  (2020-06-03T05:04:38Z)
• How Useful are Reviews for Recommendation? A Critical Review and Potential Improvements [8.471274313213092]
  We investigate a growing body of work that seeks to improve recommender systems through the use of review text. Our initial findings reveal several discrepancies in reported results, partly due to copying results across papers despite changes in experimental settings or data pre-processing. Further investigation calls for discussion on a much larger problem about the "importance" of user reviews for recommendation.
  arXiv  Detail & Related papers  (2020-05-25T16:30:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.