Explaining reputation assessments

• URL: http://arxiv.org/abs/2006.08818v1
• Date: Mon, 15 Jun 2020 23:19:35 GMT
• Title: Explaining reputation assessments
• Authors: Ingrid Nunes, Phillip Taylor, Lina Barakat, Nathan Griffiths, Simon Miles
• Abstract summary: We propose an approach to explain the rationale behind assessments from quantitative reputation models. Our approach adapts, extends and combines existing approaches for explaining decisions made using multi-attribute decision models.
• Score: 6.87724532311602
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reputation is crucial to enabling human or software agents to select among alternative providers. Although several effective reputation assessment methods exist, they typically distil reputation into a numerical representation, with no accompanying explanation of the rationale behind the assessment. Such explanations would allow users or clients to make a richer assessment of providers, and tailor selection according to their preferences and current context. In this paper, we propose an approach to explain the rationale behind assessments from quantitative reputation models, by generating arguments that are combined to form explanations. Our approach adapts, extends and combines existing approaches for explaining decisions made using multi-attribute decision models in the context of reputation. We present example argument templates, and describe how to select their parameters using explanation algorithms. Our proposal was evaluated by means of a user study, which followed an existing protocol. Our results give evidence that although explanations present a subset of the information of trust scores, they are sufficient to equally evaluate providers recommended based on their trust score. Moreover, when explanation arguments reveal implicit model information, they are less persuasive than scores.

Related papers

• Disentangling Likes and Dislikes in Personalized Generative Explainable Recommendation [26.214148426964794]
  We introduce new datasets and evaluation methods that focus on the users' sentiments. We construct the datasets by explicitly extracting users' positive and negative opinions from their post-purchase reviews. We propose to evaluate systems based on whether the generated explanations align well with the users' sentiments.
  arXiv  Detail & Related papers  (2024-10-17T06:15:00Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception [53.4840989321394]
  We analyze the effect of rationales generated by QA models to support their answers. We present users with incorrect answers and corresponding rationales in various formats. We measure the effectiveness of this feedback in patching these rationales through in-context learning.
  arXiv  Detail & Related papers  (2023-11-16T04:26:32Z)
• Using Natural Language Explanations to Rescale Human Judgments [81.66697572357477]
  We propose a method to rescale ordinal annotations and explanations using large language models (LLMs) We feed annotators' Likert ratings and corresponding explanations into an LLM and prompt it to produce a numeric score anchored in a scoring rubric. Our method rescales the raw judgments without impacting agreement and brings the scores closer to human judgments grounded in the same scoring rubric.
  arXiv  Detail & Related papers  (2023-05-24T06:19:14Z)
• Graph-based Extractive Explainer for Recommendations [38.278148661173525]
  We develop a graph attentive neural network model that seamlessly integrates user, item, attributes, and sentences for extraction-based explanation. To balance individual sentence relevance, overall attribute coverage, and content redundancy, we solve an integer linear programming problem to make the final selection of sentences.
  arXiv  Detail & Related papers  (2022-02-20T04:56:10Z)
• From Intrinsic to Counterfactual: On the Explainability of Contextualized Recommender Systems [43.93801836660617]
  We show that by utilizing the contextual features (e.g., item reviews from users), we can design a series of explainable recommender systems. We propose three types of explainable recommendation strategies with gradual change of model transparency: whitebox, graybox, and blackbox. Our model achieves highly competitive ranking performance, and generates accurate and effective explanations in terms of numerous quantitative metrics and qualitative visualizations.
  arXiv  Detail & Related papers  (2021-10-28T01:54:04Z)
• Counterfactual Explainable Recommendation [22.590877963169103]
  We propose Counterfactual Explainable Recommendation (CountER), which takes the insights of counterfactual reasoning from causal inference for explainable recommendation. CountER seeks simple (low complexity) and effective (high strength) explanations for the model decision. Results show that our model generates more accurate and effective explanations than state-of-the-art explainable recommendation models.
  arXiv  Detail & Related papers  (2021-08-24T06:37:57Z)
• Human Evaluation of Spoken vs. Visual Explanations for Open-Domain QA [22.76153284711981]
  We study whether explanations help users correctly decide when to accept or reject an ODQA system's answer. Our results show that explanations derived from retrieved evidence passages can outperform strong baselines (calibrated confidence) across modalities. We show common failure cases of current explanations, emphasize end-to-end evaluation of explanations, and caution against evaluating them in proxy modalities that are different from deployment.
  arXiv  Detail & Related papers  (2020-12-30T08:19:02Z)
• Toward Scalable and Unified Example-based Explanation and Outlier Detection [128.23117182137418]
  We argue for a broader adoption of prototype-based student networks capable of providing an example-based explanation for their prediction. We show that our prototype-based networks beyond similarity kernels deliver meaningful explanations and promising outlier detection results without compromising classification accuracy.
  arXiv  Detail & Related papers  (2020-11-11T05:58:17Z)
• Explainable Recommender Systems via Resolving Learning Representations [57.24565012731325]
  Explanations could help improve user experience and discover system defects. We propose a novel explainable recommendation model through improving the transparency of the representation learning process.
  arXiv  Detail & Related papers  (2020-08-21T05:30:48Z)
• Evaluations and Methods for Explanation through Robustness Analysis [117.7235152610957]
  We establish a novel set of evaluation criteria for such feature based explanations by analysis. We obtain new explanations that are loosely necessary and sufficient for a prediction. We extend the explanation to extract the set of features that would move the current prediction to a target class.
  arXiv  Detail & Related papers  (2020-05-31T05:52:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.