The Role of Individual User Differences in Interpretable and Explainable Machine Learning Systems

• URL: http://arxiv.org/abs/2009.06675v1
• Date: Mon, 14 Sep 2020 18:15:00 GMT
• Title: The Role of Individual User Differences in Interpretable and Explainable Machine Learning Systems
• Authors: Lydia P. Gleaves, Reva Schwartz, David A. Broniatowski
• Abstract summary: We study how individual skills and personality traits predict interpretability, explainability, and knowledge discovery from machine learning generated model output. Our work relies on Fuzzy Trace Theory, a leading theory of how humans process numerical stimuli.
• Score: 0.3169089186688223
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: There is increased interest in assisting non-expert audiences to effectively interact with machine learning (ML) tools and understand the complex output such systems produce. Here, we describe user experiments designed to study how individual skills and personality traits predict interpretability, explainability, and knowledge discovery from ML generated model output. Our work relies on Fuzzy Trace Theory, a leading theory of how humans process numerical stimuli, to examine how different end users will interpret the output they receive while interacting with the ML system. While our sample was small, we found that interpretability -- being able to make sense of system output -- and explainability -- understanding how that output was generated -- were distinct aspects of user experience. Additionally, subjects were more able to interpret model output if they possessed individual traits that promote metacognitive monitoring and editing, associated with more detailed, verbatim, processing of ML output. Finally, subjects who are more familiar with ML systems felt better supported by them and more able to discover new patterns in data; however, this did not necessarily translate to meaningful insights. Our work motivates the design of systems that explicitly take users' mental representations into account during the design process to more effectively support end user requirements.

Related papers

• Beyond Model Interpretability: Socio-Structural Explanations in Machine Learning [5.159407277301709]
  We argue that interpreting machine learning outputs in certain normatively salient domains could require appealing to a third type of explanation. The relevance of this explanation type is motivated by the fact that machine learning models are not isolated entities but are embedded within and shaped by social structures.
  arXiv  Detail & Related papers  (2024-09-05T15:47:04Z)
• From Feature Importance to Natural Language Explanations Using LLMs with RAG [4.204990010424084]
  We introduce traceable question-answering, leveraging an external knowledge repository to inform responses of Large Language Models (LLMs) This knowledge repository comprises contextual details regarding the model's output, containing high-level features, feature importance, and alternative probabilities. We integrate four key characteristics - social, causal, selective, and contrastive - drawn from social science research on human explanations into a single-shot prompt, guiding the response generation process.
  arXiv  Detail & Related papers  (2024-07-30T17:27:20Z)
• A Multimodal Automated Interpretability Agent [63.8551718480664]
  MAIA is a system that uses neural models to automate neural model understanding tasks. We first characterize MAIA's ability to describe (neuron-level) features in learned representations of images. We then show that MAIA can aid in two additional interpretability tasks: reducing sensitivity to spurious features, and automatically identifying inputs likely to be mis-classified.
  arXiv  Detail & Related papers  (2024-04-22T17:55:11Z)
• Interpreting Pretrained Language Models via Concept Bottlenecks [55.47515772358389]
  Pretrained language models (PLMs) have made significant strides in various natural language processing tasks. The lack of interpretability due to their black-box'' nature poses challenges for responsible implementation. We propose a novel approach to interpreting PLMs by employing high-level, meaningful concepts that are easily understandable for humans.
  arXiv  Detail & Related papers  (2023-11-08T20:41:18Z)
• Towards Fair and Explainable AI using a Human-Centered AI Approach [5.888646114353372]
  We present 5 research projects that aim to enhance explainability and fairness in classification systems and word embeddings. The first project explores the utility/downsides of introducing local model explanations as interfaces for machine teachers. The second project presents D-BIAS, a causality-based human-in-the-loop visual tool for identifying and mitigating social biases in datasets. The third project presents WordBias, a visual interactive tool that helps audit pre-trained static word embeddings for biases against groups. The fourth project presents DramatVis Personae, a visual analytics tool that helps identify social
  arXiv  Detail & Related papers  (2023-06-12T21:08:55Z)
• One-way Explainability Isn't The Message [2.618757282404254]
  We argue that requirements on both human and machine in this context are significantly different. The design of such human-machine systems should be driven by repeated, two-way intelligibility of information. We propose operational principles -- we call them Intelligibility Axioms -- to guide the design of a collaborative decision-support system.
  arXiv  Detail & Related papers  (2022-05-05T09:15:53Z)
• Explainable Predictive Process Monitoring: A User Evaluation [62.41400549499849]
  Explainability is motivated by the lack of transparency of black-box Machine Learning approaches. We carry on a user evaluation on explanation approaches for Predictive Process Monitoring.
  arXiv  Detail & Related papers  (2022-02-15T22:24:21Z)
• Explainable Empirical Risk Minimization [0.6299766708197883]
  Successful application of machine learning (ML) methods becomes increasingly dependent on their interpretability or explainability. This paper applies information-theoretic concepts to develop a novel measure for the subjective explainability of predictions delivered by a ML method. Our main contribution is the explainable empirical risk minimization (EERM) principle of learning a hypothesis that optimally balances between the subjective explainability and risk.
  arXiv  Detail & Related papers  (2020-09-03T07:16:34Z)
• A general framework for scientifically inspired explanations in AI [76.48625630211943]
  We instantiate the concept of structure of scientific explanation as the theoretical underpinning for a general framework in which explanations for AI systems can be implemented. This framework aims to provide the tools to build a "mental-model" of any AI system so that the interaction with the user can provide information on demand and be closer to the nature of human-made explanations.
  arXiv  Detail & Related papers  (2020-03-02T10:32:21Z)
• An Information-Theoretic Approach to Personalized Explainable Machine Learning [92.53970625312665]
  We propose a simple probabilistic model for the predictions and user knowledge. We quantify the effect of an explanation by the conditional mutual information between the explanation and prediction.
  arXiv  Detail & Related papers  (2020-03-01T13:06:29Z)
• Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience [76.9910678786031]
  We propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the recently surging field of explainable AI (XAI) into an Active Learning setting. Our study shows benefits of AI explanation as interfaces for machine teaching--supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks--anchoring effect with the model judgment and cognitive workload.
  arXiv  Detail & Related papers  (2020-01-24T22:52:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.