The Quest for the Right Mediator: A History, Survey, and Theoretical Grounding of Causal Interpretability

• URL: http://arxiv.org/abs/2408.01416v1
• Date: Fri, 2 Aug 2024 17:51:42 GMT
• Title: The Quest for the Right Mediator: A History, Survey, and Theoretical Grounding of Causal Interpretability
• Authors: Aaron Mueller, Jannik Brinkmann, Millicent Li, Samuel Marks, Koyena Pal, Nikhil Prakash, Can Rager, Aruna Sankaranarayanan, Arnab Sen Sharma, Jiuding Sun, Eric Todd, David Bau, Yonatan Belinkov,
• Abstract summary: We propose a perspective on interpretability research grounded in causal mediation analysis. We describe the history and current state of interpretability taxonomized according to the types of causal units employed. We argue that this framing yields a more cohesive narrative of the field, as well as actionable insights for future work.
• Score: 42.999302444590484
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Interpretability provides a toolset for understanding how and why neural networks behave in certain ways. However, there is little unity in the field: most studies employ ad-hoc evaluations and do not share theoretical foundations, making it difficult to measure progress and compare the pros and cons of different techniques. Furthermore, while mechanistic understanding is frequently discussed, the basic causal units underlying these mechanisms are often not explicitly defined. In this paper, we propose a perspective on interpretability research grounded in causal mediation analysis. Specifically, we describe the history and current state of interpretability taxonomized according to the types of causal units (mediators) employed, as well as methods used to search over mediators. We discuss the pros and cons of each mediator, providing insights as to when particular kinds of mediators and search methods are most appropriate depending on the goals of a given study. We argue that this framing yields a more cohesive narrative of the field, as well as actionable insights for future work. Specifically, we recommend a focus on discovering new mediators with better trade-offs between human-interpretability and compute-efficiency, and which can uncover more sophisticated abstractions from neural networks than the primarily linear mediators employed in current work. We also argue for more standardized evaluations that enable principled comparisons across mediator types, such that we can better understand when particular causal units are better suited to particular use cases.

Related papers

• Leveraging Ontologies to Document Bias in Data [1.0635248457021496]
  Doc-BiasO is a resource that aims to create an integrated vocabulary of biases defined in the textitfair-ML literature and their measures. Our main objective is to contribute towards clarifying existing terminology on bias research as it rapidly expands to all areas of AI.
  arXiv  Detail & Related papers  (2024-06-29T18:41:07Z)
• When is an Embedding Model More Promising than Another? [33.540506562970776]
  Embedders play a central role in machine learning, projecting any object into numerical representations that can be leveraged to perform various downstream tasks. The evaluation of embedding models typically depends on domain-specific empirical approaches. We present a unified approach to evaluate embedders, drawing upon the concepts of sufficiency and informativeness.
  arXiv  Detail & Related papers  (2024-06-11T18:13:46Z)
• Towards Non-Adversarial Algorithmic Recourse [20.819764720587646]
  It has been argued that adversarial examples, as opposed to counterfactual explanations, have a unique characteristic in that they lead to a misclassification compared to the ground truth. We introduce non-adversarial algorithmic recourse and outline why in high-stakes situations, it is imperative to obtain counterfactual explanations that do not exhibit adversarial characteristics.
  arXiv  Detail & Related papers  (2024-03-15T14:18:21Z)
• Fairness meets Cross-Domain Learning: a new perspective on Models and Metrics [80.07271410743806]
  We study the relationship between cross-domain learning (CD) and model fairness. We introduce a benchmark on face and medical images spanning several demographic groups as well as classification and localization tasks. Our study covers 14 CD approaches alongside three state-of-the-art fairness algorithms and shows how the former can outperform the latter.
  arXiv  Detail & Related papers  (2023-03-25T09:34:05Z)
• Investigating the Role of Centering Theory in the Context of Neural Coreference Resolution Systems [71.57556446474486]
  We investigate the connection between centering theory and modern coreference resolution systems. We show that high-quality neural coreference resolvers may not benefit much from explicitly modeling centering ideas. We formulate a version of CT that also models recency and show that it captures coreference information better compared to vanilla CT.
  arXiv  Detail & Related papers  (2022-10-26T12:55:26Z)
• Descriptive vs. inferential community detection in networks: pitfalls, myths, and half-truths [0.0]
  We argue that inferential methods are more typically aligned with clearer scientific questions, yield more robust results, and should be in many cases preferred. We attempt to dispel some myths and half-truths often believed when community detection is employed in practice, in an effort to improve both the use of such methods as well as the interpretation of their results.
  arXiv  Detail & Related papers  (2021-11-30T23:57:51Z)
• Author Clustering and Topic Estimation for Short Texts [69.54017251622211]
  We propose a novel model that expands on the Latent Dirichlet Allocation by modeling strong dependence among the words in the same document. We also simultaneously cluster users, removing the need for post-hoc cluster estimation. Our method performs as well as -- or better -- than traditional approaches to problems arising in short text.
  arXiv  Detail & Related papers  (2021-06-15T20:55:55Z)
• Prompting Contrastive Explanations for Commonsense Reasoning Tasks [74.7346558082693]
  Large pretrained language models (PLMs) can achieve near-human performance on commonsense reasoning tasks. We show how to use these same models to generate human-interpretable evidence.
  arXiv  Detail & Related papers  (2021-06-12T17:06:13Z)
• Prediction or Comparison: Toward Interpretable Qualitative Reasoning [16.02199526395448]
  Current approaches use either semantics to transform natural language inputs into logical expressions or a "black-box" model to solve them in one step. In this work, we categorize qualitative reasoning tasks into two types: prediction and comparison. In particular, we adopt neural network modules trained in an end-to-end manner to simulate the two reasoning processes.
  arXiv  Detail & Related papers  (2021-06-04T10:27:55Z)
• Individual Explanations in Machine Learning Models: A Survey for Practitioners [69.02688684221265]
  The use of sophisticated statistical models that influence decisions in domains of high societal relevance is on the rise. Many governments, institutions, and companies are reluctant to their adoption as their output is often difficult to explain in human-interpretable ways. Recently, the academic literature has proposed a substantial amount of methods for providing interpretable explanations to machine learning models.
  arXiv  Detail & Related papers  (2021-04-09T01:46:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.