Explaining reputation assessments

• URL: http://arxiv.org/abs/2006.08818v1
• Date: Mon, 15 Jun 2020 23:19:35 GMT
• Title: Explaining reputation assessments
• Authors: Ingrid Nunes, Phillip Taylor, Lina Barakat, Nathan Griffiths, Simon Miles
• Abstract summary: We propose an approach to explain the rationale behind assessments from quantitative reputation models. Our approach adapts, extends and combines existing approaches for explaining decisions made using multi-attribute decision models.
• Score: 6.87724532311602
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reputation is crucial to enabling human or software agents to select among alternative providers. Although several effective reputation assessment methods exist, they typically distil reputation into a numerical representation, with no accompanying explanation of the rationale behind the assessment. Such explanations would allow users or clients to make a richer assessment of providers, and tailor selection according to their preferences and current context. In this paper, we propose an approach to explain the rationale behind assessments from quantitative reputation models, by generating arguments that are combined to form explanations. Our approach adapts, extends and combines existing approaches for explaining decisions made using multi-attribute decision models in the context of reputation. We present example argument templates, and describe how to select their parameters using explanation algorithms. Our proposal was evaluated by means of a user study, which followed an existing protocol. Our results give evidence that although explanations present a subset of the information of trust scores, they are sufficient to equally evaluate providers recommended based on their trust score. Moreover, when explanation arguments reveal implicit model information, they are less persuasive than scores.

Related papers

• Contextualized Evaluations: Taking the Guesswork Out of Language Model Evaluations [85.81295563405433]
  Language model users often issue queries that lack specification, where the context under which a query was issued is not explicit. We present contextualized evaluations, a protocol that synthetically constructs context surrounding an under-specified query and provides it during evaluation. We find that the presence of context can 1) alter conclusions drawn from evaluation, even flipping win rates between model pairs, 2) nudge evaluators to make fewer judgments based on surface-level criteria, like style, and 3) provide new insights about model behavior across diverse contexts.
  arXiv  Detail & Related papers  (2024-11-11T18:58:38Z)
• Disentangling Likes and Dislikes in Personalized Generative Explainable Recommendation [26.214148426964794]
  We introduce new datasets and evaluation methods that focus on the users' sentiments. We construct the datasets by explicitly extracting users' positive and negative opinions from their post-purchase reviews. We propose to evaluate systems based on whether the generated explanations align well with the users' sentiments.
  arXiv  Detail & Related papers  (2024-10-17T06:15:00Z)
• Aligning Explanations for Recommendation with Rating and Feature via Maximizing Mutual Information [29.331050754362803]
  Current explanation generation methods are commonly trained with an objective to mimic existing user reviews. We propose a flexible model-agnostic method named MMI framework to enhance the alignment between the generated natural language explanations and the predicted rating/important item features. Our MMI framework can boost different backbone models, enabling them to outperform existing baselines in terms of alignment with predicted ratings and item features.
  arXiv  Detail & Related papers  (2024-07-18T08:29:55Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception [53.4840989321394]
  We analyze the effect of rationales generated by QA models to support their answers. We present users with incorrect answers and corresponding rationales in various formats. We measure the effectiveness of this feedback in patching these rationales through in-context learning.
  arXiv  Detail & Related papers  (2023-11-16T04:26:32Z)
• Using Natural Language Explanations to Rescale Human Judgments [81.66697572357477]
  We propose a method to rescale ordinal annotations and explanations using large language models (LLMs) We feed annotators' Likert ratings and corresponding explanations into an LLM and prompt it to produce a numeric score anchored in a scoring rubric. Our method rescales the raw judgments without impacting agreement and brings the scores closer to human judgments grounded in the same scoring rubric.
  arXiv  Detail & Related papers  (2023-05-24T06:19:14Z)
• Graph-based Extractive Explainer for Recommendations [38.278148661173525]
  We develop a graph attentive neural network model that seamlessly integrates user, item, attributes, and sentences for extraction-based explanation. To balance individual sentence relevance, overall attribute coverage, and content redundancy, we solve an integer linear programming problem to make the final selection of sentences.
  arXiv  Detail & Related papers  (2022-02-20T04:56:10Z)
• From Intrinsic to Counterfactual: On the Explainability of Contextualized Recommender Systems [43.93801836660617]
  We show that by utilizing the contextual features (e.g., item reviews from users), we can design a series of explainable recommender systems. We propose three types of explainable recommendation strategies with gradual change of model transparency: whitebox, graybox, and blackbox. Our model achieves highly competitive ranking performance, and generates accurate and effective explanations in terms of numerous quantitative metrics and qualitative visualizations.
  arXiv  Detail & Related papers  (2021-10-28T01:54:04Z)
• Counterfactual Explainable Recommendation [22.590877963169103]
  We propose Counterfactual Explainable Recommendation (CountER), which takes the insights of counterfactual reasoning from causal inference for explainable recommendation. CountER seeks simple (low complexity) and effective (high strength) explanations for the model decision. Results show that our model generates more accurate and effective explanations than state-of-the-art explainable recommendation models.
  arXiv  Detail & Related papers  (2021-08-24T06:37:57Z)
• Toward Scalable and Unified Example-based Explanation and Outlier Detection [128.23117182137418]
  We argue for a broader adoption of prototype-based student networks capable of providing an example-based explanation for their prediction. We show that our prototype-based networks beyond similarity kernels deliver meaningful explanations and promising outlier detection results without compromising classification accuracy.
  arXiv  Detail & Related papers  (2020-11-11T05:58:17Z)
• Evaluations and Methods for Explanation through Robustness Analysis [117.7235152610957]
  We establish a novel set of evaluation criteria for such feature based explanations by analysis. We obtain new explanations that are loosely necessary and sufficient for a prediction. We extend the explanation to extract the set of features that would move the current prediction to a target class.
  arXiv  Detail & Related papers  (2020-05-31T05:52:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.