Whom do Explanations Serve? A Systematic Literature Survey of User Characteristics in Explainable Recommender Systems Evaluation

• URL: http://arxiv.org/abs/2412.14193v2
• Date: Mon, 03 Feb 2025 16:50:32 GMT
• Title: Whom do Explanations Serve? A Systematic Literature Survey of User Characteristics in Explainable Recommender Systems Evaluation
• Authors: Kathrin Wardatzky, Oana Inel, Luca Rossetto, Abraham Bernstein,
• Abstract summary: We surveyed 124 papers in which recommender systems explanations were evaluated in user studies. Our findings suggest that the results from the surveyed studies predominantly cover specific users. We recommend actions to move toward a more inclusive and reproducible evaluation.
• Score: 7.021274080378664
• License:
• Abstract: Adding explanations to recommender systems is said to have multiple benefits, such as increasing user trust or system transparency. Previous work from other application areas suggests that specific user characteristics impact the users' perception of the explanation. However, we rarely find this type of evaluation for recommender systems explanations. This paper addresses this gap by surveying 124 papers in which recommender systems explanations were evaluated in user studies. We analyzed their participant descriptions and study results where the impact of user characteristics on the explanation effects was measured. Our findings suggest that the results from the surveyed studies predominantly cover specific users who do not necessarily represent the users of recommender systems in the evaluation domain. This may seriously hamper the generalizability of any insights we may gain from current studies on explanations in recommender systems. We further find inconsistencies in the data reporting, which impacts the reproducibility of the reported results. Hence, we recommend actions to move toward a more inclusive and reproducible evaluation.

Related papers

• Review-based Recommender Systems: A Survey of Approaches, Challenges and Future Perspectives [11.835903510784735]
  Review-based recommender systems have emerged as a significant sub-field in this domain. We present a categorization of these systems and summarize the state-of-the-art methods, analyzing their unique features, effectiveness, and limitations. We propose potential directions for future research, including the integration of multimodal data, multi-criteria rating information, and ethical considerations.
  arXiv  Detail & Related papers  (2024-05-09T05:45:18Z)
• Stability of Explainable Recommendation [10.186029242664931]
  We study the vulnerability of existent feature-oriented explainable recommenders. We observe that all the explainable models are vulnerable to increased noise levels. Our study presents an empirical verification on the topic of robust explanations in recommender systems.
  arXiv  Detail & Related papers  (2024-05-03T04:44:51Z)
• Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs [57.16442740983528]
  In ad-hoc retrieval, evaluation relies heavily on user actions, including implicit feedback. The role of user feedback in annotators' assessment of turns in a conversational perception has been little studied. We focus on how the evaluation of task-oriented dialogue systems ( TDSs) is affected by considering user feedback, explicit or implicit, as provided through the follow-up utterance of a turn being evaluated.
  arXiv  Detail & Related papers  (2024-04-19T16:45:50Z)
• The Impact of User-Level Explanation Properties on Explanation Goals in Recommender Systems [5.634769877793363]
  Explanations are crucial for improving users' transparency, persuasiveness, engagement, and trust in Recommender Systems (RSs) This paper investigates the impact of user-level explanation properties, such as diversity and popularity of attributes, on the user perception of explanation goals.
  arXiv  Detail & Related papers  (2023-10-22T18:22:35Z)
• Impression-Aware Recommender Systems [53.48892326556546]
  We present a systematic literature review on recommender systems using impressions. We define a theoretical framework to delimit recommender systems using impressions and a novel paradigm for personalized recommendations, called impression-aware recommender systems.
  arXiv  Detail & Related papers  (2023-08-15T16:16:02Z)
• User-Controllable Recommendation via Counterfactual Retrospective and Prospective Explanations [96.45414741693119]
  We present a user-controllable recommender system that seamlessly integrates explainability and controllability. By providing both retrospective and prospective explanations through counterfactual reasoning, users can customize their control over the system.
  arXiv  Detail & Related papers  (2023-08-02T01:13:36Z)
• Explainable Recommender with Geometric Information Bottleneck [25.703872435370585]
  We propose to incorporate a geometric prior learnt from user-item interactions into a variational network. Latent factors from an individual user-item pair can be used for both recommendation and explanation generation. Experimental results on three e-commerce datasets show that our model significantly improves the interpretability of a variational recommender.
  arXiv  Detail & Related papers  (2023-05-09T10:38:36Z)
• Measuring "Why" in Recommender Systems: a Comprehensive Survey on the Evaluation of Explainable Recommendation [87.82664566721917]
  This survey is based on more than 100 papers from top-tier conferences like IJCAI, AAAI, TheWebConf, Recsys, UMAP, and IUI.
  arXiv  Detail & Related papers  (2022-02-14T02:58:55Z)
• Correcting the User Feedback-Loop Bias for Recommendation Systems [34.44834423714441]
  We propose a systematic and dynamic way to correct user feedback-loop bias in recommendation systems. Our method includes a deep-learning component to learn each user's dynamic rating history embedding. We empirically validated the existence of such user feedback-loop bias in real world recommendation systems.
  arXiv  Detail & Related papers  (2021-09-13T15:02:55Z)
• Fairness-Aware Explainable Recommendation over Knowledge Graphs [73.81994676695346]
  We analyze different groups of users according to their level of activity, and find that bias exists in recommendation performance between different groups. We show that inactive users may be more susceptible to receiving unsatisfactory recommendations, due to insufficient training data for the inactive users. We propose a fairness constrained approach via re-ranking to mitigate this problem in the context of explainable recommendation over knowledge graphs.
  arXiv  Detail & Related papers  (2020-06-03T05:04:38Z)
• A Survey on Knowledge Graph-Based Recommender Systems [65.50486149662564]
  We conduct a systematical survey of knowledge graph-based recommender systems. We focus on how the papers utilize the knowledge graph for accurate and explainable recommendation. We introduce datasets used in these works.
  arXiv  Detail & Related papers  (2020-02-28T02:26:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.