Tracking Equivalent Mechanistic Interpretations Across Neural Networks

• URL: http://arxiv.org/abs/2603.30002v1
• Date: Tue, 31 Mar 2026 16:57:52 GMT
• Title: Tracking Equivalent Mechanistic Interpretations Across Neural Networks
• Abstract summary: Mechanistic interpretability (MI) is an emerging framework for interpreting neural networks.<n>We propose and formalize the principle that two interpretations of a model are equivalent if all of their possible implementations are also equivalent.<n>We provide guarantees that simultaneously relate a model's algorithmic interpretations, circuits, and representations.
• Score: 13.885265796951705
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mechanistic interpretability (MI) is an emerging framework for interpreting neural networks. Given a task and model, MI aims to discover a succinct algorithmic process, an interpretation, that explains the model's decision process on that task. However, MI is difficult to scale and generalize. This stems in part from two key challenges: there is no precise notion of a valid interpretation; and, generating interpretations is often an ad hoc process. In this paper, we address these challenges by defining and studying the problem of interpretive equivalence: determining whether two different models share a common interpretation, without requiring an explicit description of what that interpretation is. At the core of our approach, we propose and formalize the principle that two interpretations of a model are equivalent if all of their possible implementations are also equivalent. We develop an algorithm to estimate interpretive equivalence and case study its use on Transformer-based models. To analyze our algorithm, we introduce necessary and sufficient conditions for interpretive equivalence based on models' representation similarity. We provide guarantees that simultaneously relate a model's algorithmic interpretations, circuits, and representations. Our framework lays a foundation for the development of more rigorous evaluation methods of MI and automated, generalizable interpretation discovery methods.

Related papers

• The Standard Interpretable Model: A general theory of interpretable machine learning to deductively design interpretable methods using Lagrangian mechanics [37.30653132093503]
  We introduce the Standard Interpretable Model (SIM), a theory grounded in Lagrangian mechanics that enables the deductive design of interpretable methods.<n>We empirically show that the SIM identifies and solves limitations of existing methods.<n>The SIM offers pedagogical grounding for interpretability curricula and may shift the scientific community's perspective of a discipline that has long been fragmented.
  arXiv  Detail & Related papers  (2026-06-10T16:26:22Z)
• Neural Interpretable Reasoning [12.106771300842945]
  We formalize a novel modeling framework for achieving interpretability in deep learning.<n>We show that this complexity can be mitigated by treating interpretability as a Markovian property.<n>We propose a new modeling paradigm -- neural generation and interpretable execution.
  arXiv  Detail & Related papers  (2025-02-17T10:33:24Z)
• Hard to Explain: On the Computational Hardness of In-Distribution Model Interpretation [0.9558392439655016]
  The ability to interpret Machine Learning (ML) models is becoming increasingly essential. Recent work has demonstrated that it is possible to formally assess interpretability by studying the computational complexity of explaining the decisions of various models.
  arXiv  Detail & Related papers  (2024-08-07T17:20:52Z)
• Validating Mechanistic Interpretations: An Axiomatic Approach [28.336108192282737]
  We give a set of axioms that formally characterize a mechanistic interpretation as a description that approximately captures the semantics of the neural network under analysis in a compositional manner.<n>We demonstrate the applicability of these axioms for validating mechanistic interpretations on an existing, well-known interpretability study as well as on a new case study involving a Transformer-based model trained to solve the well-known 2-SAT problem.
  arXiv  Detail & Related papers  (2024-07-18T15:32:44Z)
• Selecting Interpretability Techniques for Healthcare Machine Learning models [69.65384453064829]
  In healthcare there is a pursuit for employing interpretable algorithms to assist healthcare professionals in several decision scenarios. We overview a selection of eight algorithms, both post-hoc and model-based, that can be used for such purposes.
  arXiv  Detail & Related papers  (2024-06-14T17:49:04Z)
• FICNN: A Framework for the Interpretation of Deep Convolutional Neural Networks [0.0]
  The aim of this paper is to propose a framework for the study of interpretation methods designed for CNN models trained from visual data. Our framework highlights that just a very small amount of the suggested factors, and combinations thereof, have been actually studied.
  arXiv  Detail & Related papers  (2023-05-17T10:59:55Z)
• Evaluating the Robustness of Interpretability Methods through Explanation Invariance and Equivariance [72.50214227616728]
  Interpretability methods are valuable only if their explanations faithfully describe the explained model. We consider neural networks whose predictions are invariant under a specific symmetry group.
  arXiv  Detail & Related papers  (2023-04-13T17:59:03Z)
• Neural Causal Models for Counterfactual Identification and Estimation [62.30444687707919]
  We study the evaluation of counterfactual statements through neural models. First, we show that neural causal models (NCMs) are expressive enough. Second, we develop an algorithm for simultaneously identifying and estimating counterfactual distributions.
  arXiv  Detail & Related papers  (2022-09-30T18:29:09Z)
• On the Faithfulness Measurements for Model Interpretations [100.2730234575114]
  Post-hoc interpretations aim to uncover how natural language processing (NLP) models make predictions. To tackle these issues, we start with three criteria: the removal-based criterion, the sensitivity of interpretations, and the stability of interpretations. Motivated by the desideratum of these faithfulness notions, we introduce a new class of interpretation methods that adopt techniques from the adversarial domain.
  arXiv  Detail & Related papers  (2021-04-18T09:19:44Z)
• Interpretable Deep Learning: Interpretations, Interpretability, Trustworthiness, and Beyond [49.93153180169685]
  We introduce and clarify two basic concepts-interpretations and interpretability-that people usually get confused. We elaborate the design of several recent interpretation algorithms, from different perspectives, through proposing a new taxonomy. We summarize the existing work in evaluating models' interpretability using "trustworthy" interpretation algorithms.
  arXiv  Detail & Related papers  (2021-03-19T08:40:30Z)
• Understanding Interpretability by generalized distillation in Supervised Classification [3.5473853445215897]
  Recent interpretation strategies focus on human understanding of the underlying decision mechanisms of the complex Machine Learning models. We propose an interpretation-by-distillation formulation that is defined relative to other ML models. We evaluate our proposed framework on the MNIST, Fashion-MNIST and Stanford40 datasets.
  arXiv  Detail & Related papers  (2020-12-05T17:42:50Z)
• Interpretable Multi-dataset Evaluation for Named Entity Recognition [110.64368106131062]
  We present a general methodology for interpretable evaluation for the named entity recognition (NER) task. The proposed evaluation method enables us to interpret the differences in models and datasets, as well as the interplay between them. By making our analysis tool available, we make it easy for future researchers to run similar analyses and drive progress in this area.
  arXiv  Detail & Related papers  (2020-11-13T10:53:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.