In Defence of Post-hoc Explainability

• URL: http://arxiv.org/abs/2412.17883v1
• Date: Mon, 23 Dec 2024 06:22:03 GMT
• Title: In Defence of Post-hoc Explainability
• Authors: Nick Oh,
• Abstract summary: We introduce Computational Interpretabilism (CI) as a philosophical framework for post-hoc interpretability in scientific AI. Drawing parallels with human expertise, where post-hoc rationalisation coexists with reliable performance, CI establishes that scientific knowledge emerges through structured model interpretation when properly bounded by empirical validation.
• Score: 0.0
• License:
• Abstract: The widespread adoption of machine learning in scientific research has created a fundamental tension between model opacity and scientific understanding. Whilst some advocate for intrinsically interpretable models, we introduce Computational Interpretabilism (CI) as a philosophical framework for post-hoc interpretability in scientific AI. Drawing parallels with human expertise, where post-hoc rationalisation coexists with reliable performance, CI establishes that scientific knowledge emerges through structured model interpretation when properly bounded by empirical validation. Through mediated understanding and bounded factivity, we demonstrate how post-hoc methods achieve epistemically justified insights without requiring complete mechanical transparency, resolving tensions between model complexity and scientific comprehension.

Related papers

• Towards a Formal Theory of the Need for Competence via Computational Intrinsic Motivation [6.593505830504729]
  We focus on the "need for competence", postulated as a key basic human need within Self-Determination Theory (SDT) We propose that these inconsistencies may be alleviated by drawing on computational models from the field of reinforcement learning (RL) Our work can support a cycle of theory development by inspiring new computational models formalising aspects of the theory, which can then be tested empirically to refine the theory.
  arXiv  Detail & Related papers  (2025-02-11T10:03:40Z)
• Causal Representation Learning from Multimodal Biological Observations [57.00712157758845]
  We aim to develop flexible identification conditions for multimodal data. We establish identifiability guarantees for each latent component, extending the subspace identification results from prior work. Our key theoretical ingredient is the structural sparsity of the causal connections among distinct modalities.
  arXiv  Detail & Related papers  (2024-11-10T16:40:27Z)
• Learning Discrete Concepts in Latent Hierarchical Models [73.01229236386148]
  Learning concepts from natural high-dimensional data holds potential in building human-aligned and interpretable machine learning models. We formalize concepts as discrete latent causal variables that are related via a hierarchical causal model. We substantiate our theoretical claims with synthetic data experiments.
  arXiv  Detail & Related papers  (2024-06-01T18:01:03Z)
• LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery [141.39722070734737]
  We propose to enhance the knowledge-driven, abstract reasoning abilities of Large Language Models with the computational strength of simulations. We introduce Scientific Generative Agent (SGA), a bilevel optimization framework. We conduct experiments to demonstrate our framework's efficacy in law discovery and molecular design.
  arXiv  Detail & Related papers  (2024-05-16T03:04:10Z)
• Diverse Explanations From Data-Driven and Domain-Driven Perspectives in the Physical Sciences [4.442043151145212]
  This Perspective explores the sources and implications of diverse explanations in machine learning applications for physical sciences. We examine how different models, explanation methods, levels of feature attribution, and stakeholder needs can result in varying interpretations of ML outputs. Our analysis underscores the importance of considering multiple perspectives when interpreting ML models in scientific contexts.
  arXiv  Detail & Related papers  (2024-02-01T05:28:28Z)
• Approach to Toric Code Anyon Excitation, Indirect Effects of Kitaev Spin in Local Social Opinion Models [0.0]
  Recent research integrates quantum information theory concepts, such as Graph States, Stabilizer States, and Toric Codes. These applications face challenges in complexity, interpretation, and empirical validation. Despite its potential, the practical use of the Toric Code Hamiltonian in Opinion Dynamics requires further exploration and research.
  arXiv  Detail & Related papers  (2023-12-02T20:25:42Z)
• Targeted Reduction of Causal Models [55.11778726095353]
  Causal Representation Learning offers a promising avenue to uncover interpretable causal patterns in simulations. We introduce Targeted Causal Reduction (TCR), a method for condensing complex intervenable models into a concise set of causal factors. Its ability to generate interpretable high-level explanations from complex models is demonstrated on toy and mechanical systems.
  arXiv  Detail & Related papers  (2023-11-30T15:46:22Z)
• AI Research Associate for Early-Stage Scientific Discovery [1.6861004263551447]
  Artificial intelligence (AI) has been increasingly applied in scientific activities for decades. We present an AI research associate for early-stage scientific discovery based on a novel minimally-biased physics-based modeling.
  arXiv  Detail & Related papers  (2022-02-02T17:05:52Z)
• Neuro-symbolic Architectures for Context Understanding [59.899606495602406]
  We propose the use of hybrid AI methodology as a framework for combining the strengths of data-driven and knowledge-driven approaches. Specifically, we inherit the concept of neuro-symbolism as a way of using knowledge-bases to guide the learning progress of deep neural networks.
  arXiv  Detail & Related papers  (2020-03-09T15:04:07Z)
• A general framework for scientifically inspired explanations in AI [76.48625630211943]
  We instantiate the concept of structure of scientific explanation as the theoretical underpinning for a general framework in which explanations for AI systems can be implemented. This framework aims to provide the tools to build a "mental-model" of any AI system so that the interaction with the user can provide information on demand and be closer to the nature of human-made explanations.
  arXiv  Detail & Related papers  (2020-03-02T10:32:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.