Causal Abstraction in Model Interpretability: A Compact Survey

• URL: http://arxiv.org/abs/2410.20161v1
• Date: Sat, 26 Oct 2024 12:24:28 GMT
• Title: Causal Abstraction in Model Interpretability: A Compact Survey
• Authors: Yihao Zhang,
• Abstract summary: causal abstraction provides a principled approach to understanding and explaining the causal mechanisms underlying model behavior. This survey paper delves into the realm of causal abstraction, examining its theoretical foundations, practical applications, and implications for the field of model interpretability.
• Score: 5.963324728136442
• License:
• Abstract: The pursuit of interpretable artificial intelligence has led to significant advancements in the development of methods that aim to explain the decision-making processes of complex models, such as deep learning systems. Among these methods, causal abstraction stands out as a theoretical framework that provides a principled approach to understanding and explaining the causal mechanisms underlying model behavior. This survey paper delves into the realm of causal abstraction, examining its theoretical foundations, practical applications, and implications for the field of model interpretability.

Related papers

• Learning Discrete Concepts in Latent Hierarchical Models [73.01229236386148]
  Learning concepts from natural high-dimensional data holds potential in building human-aligned and interpretable machine learning models. We formalize concepts as discrete latent causal variables that are related via a hierarchical causal model. We substantiate our theoretical claims with synthetic data experiments.
  arXiv  Detail & Related papers  (2024-06-01T18:01:03Z)
• Inverse Decision Modeling: Learning Interpretable Representations of Behavior [72.80902932543474]
  We develop an expressive, unifying perspective on inverse decision modeling. We use this to formalize the inverse problem (as a descriptive model) We illustrate how this structure enables learning (interpretable) representations of (bounded) rationality.
  arXiv  Detail & Related papers  (2023-10-28T05:05:01Z)
• Explainability for Large Language Models: A Survey [59.67574757137078]
  Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. This paper introduces a taxonomy of explainability techniques and provides a structured overview of methods for explaining Transformer-based language models.
  arXiv  Detail & Related papers  (2023-09-02T22:14:26Z)
• Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability [30.76910454663951]
  Causal abstraction provides a theoretical foundation for mechanistic interpretability. Our contributions are generalizing the theory of causal abstraction from mechanism replacement to arbitrary mechanism transformation.
  arXiv  Detail & Related papers  (2023-01-11T20:42:41Z)
• Abstract Interpretation for Generalized Heuristic Search in Model-Based Planning [50.96320003643406]
  Domain-general model-based planners often derive their generality by constructing searchs through the relaxation of symbolic world models. We illustrate how abstract interpretation can serve as a unifying framework for these abstractions, extending the reach of search to richer world models. Theses can also be integrated with learning, allowing agents to jumpstart planning in novel world models via abstraction-derived information.
  arXiv  Detail & Related papers  (2022-08-05T00:22:11Z)
• Towards Computing an Optimal Abstraction for Structural Causal Models [16.17846886492361]
  We focus on the problem of learning abstractions. We suggest a concrete measure of information loss, and we illustrate its contribution to learning new abstractions.
  arXiv  Detail & Related papers  (2022-08-01T14:35:57Z)
• Causal Dynamics Learning for Task-Independent State Abstraction [61.707048209272884]
  We introduce Causal Dynamics Learning for Task-Independent State Abstraction (CDL) CDL learns a theoretically proved causal dynamics model that removes unnecessary dependencies between state variables and the action. A state abstraction can then be derived from the learned dynamics.
  arXiv  Detail & Related papers  (2022-06-27T17:02:53Z)
• Towards Interpretable Reasoning over Paragraph Effects in Situation [126.65672196760345]
  We focus on the task of reasoning over paragraph effects in situation, which requires a model to understand the cause and effect. We propose a sequential approach for this task which explicitly models each step of the reasoning process with neural network modules. In particular, five reasoning modules are designed and learned in an end-to-end manner, which leads to a more interpretable model.
  arXiv  Detail & Related papers  (2020-10-03T04:03:52Z)
• Knowledge Patterns [19.57676317580847]
  This paper describes a new technique, called "knowledge patterns", for helping construct axiom-rich, formal Ontology. Knowledge patterns provide an important insight into the structure of a formal Ontology. We describe the technique and an application built using them, and then critique their strengths and weaknesses.
  arXiv  Detail & Related papers  (2020-05-08T22:33:30Z)
• The Pragmatic Turn in Explainable Artificial Intelligence (XAI) [0.0]
  I argue that the search for explainable models and interpretable decisions in AI must be reformulated in terms of the broader project of offering a pragmatic and naturalistic account of understanding in AI. I conclude that interpretative or approximation models not only provide the best way to achieve the objectual understanding of a machine learning model, but are also a necessary condition to achieve post-hoc interpretability.
  arXiv  Detail & Related papers  (2020-02-22T01:40:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.