Beyond Model Interpretability: Socio-Structural Explanations in Machine Learning

• URL: http://arxiv.org/abs/2409.03632v1
• Date: Thu, 5 Sep 2024 15:47:04 GMT
• Title: Beyond Model Interpretability: Socio-Structural Explanations in Machine Learning
• Authors: Andrew Smart, Atoosa Kasirzadeh,
• Abstract summary: We argue that interpreting machine learning outputs in certain normatively salient domains could require appealing to a third type of explanation. The relevance of this explanation type is motivated by the fact that machine learning models are not isolated entities but are embedded within and shaped by social structures.
• Score: 5.159407277301709
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: What is it to interpret the outputs of an opaque machine learning model. One approach is to develop interpretable machine learning techniques. These techniques aim to show how machine learning models function by providing either model centric local or global explanations, which can be based on mechanistic interpretations revealing the inner working mechanisms of models or nonmechanistic approximations showing input feature output data relationships. In this paper, we draw on social philosophy to argue that interpreting machine learning outputs in certain normatively salient domains could require appealing to a third type of explanation that we call sociostructural explanation. The relevance of this explanation type is motivated by the fact that machine learning models are not isolated entities but are embedded within and shaped by social structures. Sociostructural explanations aim to illustrate how social structures contribute to and partially explain the outputs of machine learning models. We demonstrate the importance of sociostructural explanations by examining a racially biased healthcare allocation algorithm. Our proposal highlights the need for transparency beyond model interpretability, understanding the outputs of machine learning systems could require a broader analysis that extends beyond the understanding of the machine learning model itself.

Related papers

• Explaining Text Similarity in Transformer Models [52.571158418102584]
  Recent advances in explainable AI have made it possible to mitigate limitations by leveraging improved explanations for Transformers. We use BiLRP, an extension developed for computing second-order explanations in bilinear similarity models, to investigate which feature interactions drive similarity in NLP models. Our findings contribute to a deeper understanding of different semantic similarity tasks and models, highlighting how novel explainable AI methods enable in-depth analyses and corpus-level insights.
  arXiv  Detail & Related papers  (2024-05-10T17:11:31Z)
• LVLM-Interpret: An Interpretability Tool for Large Vision-Language Models [50.259006481656094]
  We present a novel interactive application aimed towards understanding the internal mechanisms of large vision-language models. Our interface is designed to enhance the interpretability of the image patches, which are instrumental in generating an answer. We present a case study of how our application can aid in understanding failure mechanisms in a popular large multi-modal model: LLaVA.
  arXiv  Detail & Related papers  (2024-04-03T23:57:34Z)
• Binding Dynamics in Rotating Features [72.80071820194273]
  We propose an alternative "cosine binding" mechanism, which explicitly computes the alignment between features and adjusts weights accordingly. This allows us to draw direct connections to self-attention and biological neural processes, and to shed light on the fundamental dynamics for object-centric representations to emerge in Rotating Features.
  arXiv  Detail & Related papers  (2024-02-08T12:31:08Z)
• Explainability for Large Language Models: A Survey [59.67574757137078]
  Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. This paper introduces a taxonomy of explainability techniques and provides a structured overview of methods for explaining Transformer-based language models.
  arXiv  Detail & Related papers  (2023-09-02T22:14:26Z)
• Explainability in Machine Learning: a Pedagogical Perspective [9.393988089692947]
  We provide a pedagogical perspective on how to structure the learning process to better impart knowledge to students and researchers in machine learning. We discuss the advantages and disadvantages of various opaque and transparent machine learning models. We will also discuss ways to structure potential assignments to best help students learn to use explainability as a tool alongside any given machine learning application.
  arXiv  Detail & Related papers  (2022-02-21T16:15:57Z)
• Properties from Mechanisms: An Equivariance Perspective on Identifiable Representation Learning [79.4957965474334]
  Key goal of unsupervised representation learning is "inverting" a data generating process to recover its latent properties. This paper asks, "Can we instead identify latent properties by leveraging knowledge of the mechanisms that govern their evolution?" We provide a complete characterization of the sources of non-identifiability as we vary knowledge about a set of possible mechanisms.
  arXiv  Detail & Related papers  (2021-10-29T14:04:08Z)
• Explainable Machine Learning with Prior Knowledge: An Overview [1.1045760002858451]
  The complexity of machine learning models has elicited research to make them more explainable. We propose to harness prior knowledge to improve upon the explanation capabilities of machine learning models.
  arXiv  Detail & Related papers  (2021-05-21T07:33:22Z)
• Counterfactual Explanations for Machine Learning: A Review [5.908471365011942]
  We review and categorize research on counterfactual explanations in machine learning. Modern approaches to counterfactual explainability in machine learning draw connections to the established legal doctrine in many countries.
  arXiv  Detail & Related papers  (2020-10-20T20:08:42Z)
• A general framework for scientifically inspired explanations in AI [76.48625630211943]
  We instantiate the concept of structure of scientific explanation as the theoretical underpinning for a general framework in which explanations for AI systems can be implemented. This framework aims to provide the tools to build a "mental-model" of any AI system so that the interaction with the user can provide information on demand and be closer to the nature of human-made explanations.
  arXiv  Detail & Related papers  (2020-03-02T10:32:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.