Interpreting "Interpretability" and Explaining "Explainability" in Machine Learning in Physics

• URL: http://arxiv.org/abs/2606.26228v1
• Date: Wed, 24 Jun 2026 18:00:01 GMT
• Title: Interpreting "Interpretability" and Explaining "Explainability" in Machine Learning in Physics
• Abstract summary: We review the concepts of interpretability and explainability as they apply to machine learning in physics.<n>We emphasize that machine-learned models are subject to the same scientific questions as classical models, differing only in scale.
• Score: 0.6372261626436676
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We review the concepts of interpretability and explainability as they apply to machine learning in physics. We define interpretability as concerning the structural transparency of a model (the ability to understand or approximate its inner workings) and explainability as concerning the scientific content of a model (the ability to map it onto domain knowledge). We discuss the trade-offs each entails (interpretability vs. expressivity; explainability vs. adaptability), the contexts in which each is needed, and the intrinsic and post-hoc tools available for achieving them. Throughout, we emphasize that machine-learned models are subject to the same scientific questions as classical models, differing only in scale, and that interpretability and explainability are best understood as deliberate modeling choices rather than inherent properties. We also emphasize the importance of task specification and intervention plans as a core aspect of model design.

Related papers

• From Observation to Insight: Mechanistic World Models and the Quest for Autonomous Discovery [60.11725132433485]
  We argue that scientific discovery is fundamentally a problem of knowledge organisation.<n>We introduce Mechanistic World Models, a new design paradigm that places reusable mechanisms at the centre of representation, computation and learning.
  arXiv  Detail & Related papers  (2026-07-14T08:03:27Z)
• From Black-box to Causal-box: Towards Building More Interpretable Models [57.23201263629627]
  We introduce the notion of causal interpretability, which formalizes when counterfactual queries can be evaluated from a specific class of models.<n>We derive a complete graphical criterion that determines whether a given model architecture supports a given counterfactual query.
  arXiv  Detail & Related papers  (2025-10-24T20:03:18Z)
• A Mathematical Philosophy of Explanations in Mechanistic Interpretability -- The Strange Science Part I.i [0.0]
  We argue that Mechanistic Interpretability research is a principled approach to understanding models.<n>We show that Explanatory Faithfulness, an assessment of how well an explanation fits a model, is well-defined.
  arXiv  Detail & Related papers  (2025-05-01T19:08:34Z)
• Beyond Model Interpretability: Socio-Structural Explanations in Machine Learning [5.159407277301709]
  We argue that interpreting machine learning outputs in certain normatively salient domains could require appealing to a third type of explanation. The relevance of this explanation type is motivated by the fact that machine learning models are not isolated entities but are embedded within and shaped by social structures.
  arXiv  Detail & Related papers  (2024-09-05T15:47:04Z)
• On the Relationship Between Interpretability and Explainability in Machine Learning [2.828173677501078]
  Interpretability and explainability have gained more and more attention in the field of machine learning. Since both provide information about predictors and their decision process, they are often seen as two independent means for one single end. This view has led to a dichotomous literature: explainability techniques designed for complex black-box models, or interpretable approaches ignoring the many explainability tools.
  arXiv  Detail & Related papers  (2023-11-20T02:31:08Z)
• Explainability for Large Language Models: A Survey [59.67574757137078]
  Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. This paper introduces a taxonomy of explainability techniques and provides a structured overview of methods for explaining Transformer-based language models.
  arXiv  Detail & Related papers  (2023-09-02T22:14:26Z)
• ExSum: From Local Explanations to Model Understanding [6.23934576145261]
  Interpretability methods are developed to understand the working mechanisms of black-box models. Fulfilling this goal requires both that the explanations generated by these methods are correct and that people can easily and reliably understand them. We introduce explanation summary (ExSum), a mathematical framework for quantifying model understanding.
  arXiv  Detail & Related papers  (2022-04-30T02:07:20Z)
• Explainability in Process Outcome Prediction: Guidelines to Obtain Interpretable and Faithful Models [77.34726150561087]
  We define explainability through the interpretability of the explanations and the faithfulness of the explainability model in the field of process outcome prediction. This paper contributes a set of guidelines named X-MOP which allows selecting the appropriate model based on the event log specifications.
  arXiv  Detail & Related papers  (2022-03-30T05:59:50Z)
• The Need for Interpretable Features: Motivation and Taxonomy [69.07189753428553]
  We claim that the term "interpretable feature" is not specific nor detailed enough to capture the full extent to which features impact the usefulness of machine learning explanations. In this paper, we motivate and discuss three key lessons: 1) more attention should be given to what we refer to as the interpretable feature space, or the state of features that are useful to domain experts taking real-world actions.
  arXiv  Detail & Related papers  (2022-02-23T19:19:14Z)
• Rethinking Explainability as a Dialogue: A Practitioner's Perspective [57.87089539718344]
  We ask doctors, healthcare professionals, and policymakers about their needs and desires for explanations. Our study indicates that decision-makers would strongly prefer interactive explanations in the form of natural language dialogues. Considering these needs, we outline a set of five principles researchers should follow when designing interactive explanations.
  arXiv  Detail & Related papers  (2022-02-03T22:17:21Z)
• The Pragmatic Turn in Explainable Artificial Intelligence (XAI) [0.0]
  I argue that the search for explainable models and interpretable decisions in AI must be reformulated in terms of the broader project of offering a pragmatic and naturalistic account of understanding in AI. I conclude that interpretative or approximation models not only provide the best way to achieve the objectual understanding of a machine learning model, but are also a necessary condition to achieve post-hoc interpretability.
  arXiv  Detail & Related papers  (2020-02-22T01:40:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.