Interpretability Can Be Actionable

• URL: http://arxiv.org/abs/2605.11161v1
• Date: Mon, 11 May 2026 19:08:21 GMT
• Title: Interpretability Can Be Actionable
• Abstract summary: Interpretability aims to explain the behavior of deep neural networks.<n>Despite rapid growth, there is mounting concern that much of this work has not translated into practical impact.<n>This position paper argues that the central missing ingredient is not new methods, but evaluation criteria: interpretability should be evaluated by actionability.
• Score: 50.72539106927059
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Interpretability aims to explain the behavior of deep neural networks. Despite rapid growth, there is mounting concern that much of this work has not translated into practical impact, raising questions about its relevance and utility. This position paper argues that the central missing ingredient is not new methods, but evaluation criteria: interpretability should be evaluated by actionability--the extent to which insights enable concrete decisions and interventions beyond interpretability research itself. We define actionable interpretability along two dimensions--concreteness and validation--and analyze the barriers currently preventing real-world impact. To address these barriers, we identify five domains where interpretability offers unique leverage and present a framework for actionable interpretability with evaluation criteria aligned with practical outcomes. Our goal is not to downplay exploratory research, but to establish actionability as a core objective of interpretability research.

Related papers

• Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal Structure [58.89643769707751]
  We study latent chain-of-thought as a manipulable causal process in representation space.<n>We find that latent-step budgets behave less like homogeneous extra depth and more like staged functionality with non-local routing.<n>These results motivate mode-conditional and stability-aware analyses as more reliable tools for interpreting and improving latent reasoning systems.
  arXiv  Detail & Related papers  (2026-02-09T15:25:12Z)
• Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process [66.38541693477181]
  We propose an unsupervised framework for discovering reasoning vectors, which we define as directions in the activation space that encode distinct reasoning behaviors.<n>By segmenting chain-of-thought traces into sentence-level'steps', we uncover disentangled features corresponding to interpretable behaviors such as reflection and backtracking.<n>We demonstrate the ability to control response confidence by identifying confidence-related vectors in the SAE decoder space.
  arXiv  Detail & Related papers  (2025-12-30T05:09:11Z)
• "There Is No Such Thing as a Dumb Question," But There Are Good Ones [4.962252439662465]
  This study defines good questions and presents a systematic evaluation framework.<n>We propose two key evaluation dimensions: appropriateness (sociolinguistic competence in context) and effectiveness.<n>By incorporating dynamic contextual variables, our evaluation framework achieves structure and flexibility through semi-adaptive criteria.
  arXiv  Detail & Related papers  (2025-05-15T03:12:28Z)
• Reasoner Outperforms: Generative Stance Detection with Rationalization for Social Media [12.479554210753664]
  This study adopts a generative approach, where stance predictions include explicit, interpretable rationales.<n>We find that incorporating reasoning into stance detection enables the smaller model (FlanT5) to outperform GPT-3.5's zero-shot performance.
  arXiv  Detail & Related papers  (2024-12-13T16:34:39Z)
• A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation [0.0]
  This review consolidates and contextualizes foundational concepts in uncertainty modeling.<n>We identify challenges, such as strong assumptions in spatial aggregation and lack of standardized benchmarks.<n>We propose directions for advancing uncertainty-aware segmentation in deep learning.
  arXiv  Detail & Related papers  (2024-11-25T13:26:09Z)
• Intrinsic User-Centric Interpretability through Global Mixture of Experts [31.738009841932374]
  InterpretCC is a family of intrinsically interpretable neural networks that optimize for ease of human understanding and explanation faithfulness.<n>We show that InterpretCC explanations are found to have higher actionability and usefulness over other intrinsically interpretable approaches.
  arXiv  Detail & Related papers  (2024-02-05T11:55:50Z)
• Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs [58.620269228776294]
  We propose a task-agnostic framework for resolving ambiguity by asking users clarifying questions. We evaluate systems across three NLP applications: question answering, machine translation and natural language inference. We find that intent-sim is robust, demonstrating improvements across a wide range of NLP tasks and LMs.
  arXiv  Detail & Related papers  (2023-11-16T00:18:50Z)
• Fidelity of Interpretability Methods and Perturbation Artifacts in Neural Networks [5.439020425819001]
  Post-hoc interpretability methods aim to quantify the importance of input features with respect to the class probabilities. A popular approach to evaluate interpretability methods is to perturb input features deemed important for a given prediction and observe the decrease in accuracy. We propose a method for estimating the impact of such artifacts on the fidelity estimation by utilizing model accuracy curves from perturbing input features.
  arXiv  Detail & Related papers  (2022-03-06T10:14:09Z)
• Empirical Estimates on Hand Manipulation are Recoverable: A Step Towards Individualized and Explainable Robotic Support in Everyday Activities [80.37857025201036]
  Key challenge for robotic systems is to figure out the behavior of another agent. Processing correct inferences is especially challenging when (confounding) factors are not controlled experimentally. We propose equipping robots with the necessary tools to conduct observational studies on people.
  arXiv  Detail & Related papers  (2022-01-27T22:15:56Z)
• On the Faithfulness Measurements for Model Interpretations [100.2730234575114]
  Post-hoc interpretations aim to uncover how natural language processing (NLP) models make predictions. To tackle these issues, we start with three criteria: the removal-based criterion, the sensitivity of interpretations, and the stability of interpretations. Motivated by the desideratum of these faithfulness notions, we introduce a new class of interpretation methods that adopt techniques from the adversarial domain.
  arXiv  Detail & Related papers  (2021-04-18T09:19:44Z)
• A Bayesian Account of Measures of Interpretability in Human-AI Interaction [34.99424576619341]
  Existing approaches for the design of interpretable agent behavior consider different measures of interpretability in isolation. We propose a revised model where all these behaviors can be meaningfully modeled together. We will highlight interesting consequences of this unified model and motivate, through results of a user study.
  arXiv  Detail & Related papers  (2020-11-22T03:28:28Z)
• Towards falsifiable interpretability research [7.360807642941714]
  We argue that interpretability research suffers from an over-reliance on intuition-based approaches. We examine two popular classes of interpretability methods-saliency and single-neuron-based approaches. We propose a strategy to address these impediments in the form of a framework for strongly falsifiable interpretability research.
  arXiv  Detail & Related papers  (2020-10-22T22:03:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.