Pathwise Explanation of ReLU Neural Networks

• URL: http://arxiv.org/abs/2506.18037v1
• Date: Sun, 22 Jun 2025 13:41:42 GMT
• Title: Pathwise Explanation of ReLU Neural Networks
• Authors: Seongwoo Lim, Won Jo, Joohyung Lee, Jaesik Choi,
• Abstract summary: We introduce a novel approach that considers subsets of the hidden units involved in the decision making path.<n>This pathwise explanation provides a clearer and more consistent understanding of the relationship between the input and the decision-making process.
• Score: 20.848391252661074
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural networks have demonstrated a wide range of successes, but their ``black box" nature raises concerns about transparency and reliability. Previous research on ReLU networks has sought to unwrap these networks into linear models based on activation states of all hidden units. In this paper, we introduce a novel approach that considers subsets of the hidden units involved in the decision making path. This pathwise explanation provides a clearer and more consistent understanding of the relationship between the input and the decision-making process. Our method also offers flexibility in adjusting the range of explanations within the input, i.e., from an overall attribution input to particular components within the input. Furthermore, it allows for the decomposition of explanations for a given input for more detailed explanations. Experiments demonstrate that our method outperforms others both quantitatively and qualitatively.

Related papers

• Concept-Guided Interpretability via Neural Chunking [54.73787666584143]
  We show that neural networks exhibit patterns in their raw population activity that mirror regularities in the training data.<n>We propose three methods to extract these emerging entities, complementing each other based on label availability and dimensionality.<n>Our work points to a new direction for interpretability, one that harnesses both cognitive principles and the structure of naturalistic data.
  arXiv  Detail & Related papers  (2025-05-16T13:49:43Z)
• Perturbation on Feature Coalition: Towards Interpretable Deep Neural Networks [0.1398098625978622]
  The "black box" nature of deep neural networks (DNNs) compromises their transparency and reliability. We introduce a perturbation-based interpretation guided by feature coalitions, which leverages deep information of network to extract correlated features.
  arXiv  Detail & Related papers  (2024-08-23T22:44:21Z)
• Network Inversion of Convolutional Neural Nets [3.004632712148892]
  Neural networks have emerged as powerful tools across various applications, yet their decision-making process often remains opaque. Network inversion techniques offer a solution by allowing us to peek inside these black boxes. This paper presents a simple yet effective approach to network inversion using a meticulously conditioned generator.
  arXiv  Detail & Related papers  (2024-07-25T12:53:21Z)
• Manipulating Feature Visualizations with Gradient Slingshots [53.94925202421929]
  Feature Visualization (FV) is a widely used technique for interpreting the concepts learned by Deep Neural Networks (DNNs)<n>We introduce a novel method, Gradient Slingshots, that enables manipulation of FV without modifying the model architecture or significantly degrading its performance.
  arXiv  Detail & Related papers  (2024-01-11T18:57:17Z)
• Towards Better Visualizing the Decision Basis of Networks via Unfold and Conquer Attribution Guidance [29.016425469068587]
  We propose a novel framework, Unfold and Conquer Guidance (UCAG), which enhances the explainability of the network decision. UCAG sequentially complies with the confidence of slices of the image, leading to providing an abundant and clear interpretation. We conduct numerous evaluations to validate the performance in several metrics.
  arXiv  Detail & Related papers  (2023-12-21T03:43:19Z)
• Shap-CAM: Visual Explanations for Convolutional Neural Networks based on Shapley Value [86.69600830581912]
  We develop a novel visual explanation method called Shap-CAM based on class activation mapping. We demonstrate that Shap-CAM achieves better visual performance and fairness for interpreting the decision making process.
  arXiv  Detail & Related papers  (2022-08-07T00:59:23Z)
• KAT: A Knowledge Augmented Transformer for Vision-and-Language [56.716531169609915]
  We propose a novel model - Knowledge Augmented Transformer (KAT) - which achieves a strong state-of-the-art result on the open-domain multimodal task of OK-VQA. Our approach integrates implicit and explicit knowledge in an end to end encoder-decoder architecture, while still jointly reasoning over both knowledge sources during answer generation. An additional benefit of explicit knowledge integration is seen in improved interpretability of model predictions in our analysis.
  arXiv  Detail & Related papers  (2021-12-16T04:37:10Z)
• How Much Can I Trust You? -- Quantifying Uncertainties in Explaining Neural Networks [19.648814035399013]
  Explainable AI (XAI) aims to provide interpretations for predictions made by learning machines, such as deep neural networks. We propose a new framework that allows to convert any arbitrary explanation method for neural networks into an explanation method for Bayesian neural networks. We demonstrate the effectiveness and usefulness of our approach extensively in various experiments.
  arXiv  Detail & Related papers  (2020-06-16T08:54:42Z)
• Explainable Deep Classification Models for Domain Generalization [94.43131722655617]
  Explanations are defined as regions of visual evidence upon which a deep classification network makes a decision. Our training strategy enforces a periodic saliency-based feedback to encourage the model to focus on the image regions that directly correspond to the ground-truth object.
  arXiv  Detail & Related papers  (2020-03-13T22:22:15Z)
• Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations [143.3053365553897]
  We describe a procedure for removing dependency on a cohort of training data from a trained deep network. We introduce a new bound on how much information can be extracted per query about the forgotten cohort. We exploit the connections between the activation and weight dynamics of a DNN inspired by Neural Tangent Kernels to compute the information in the activations.
  arXiv  Detail & Related papers  (2020-03-05T23:17:35Z)
• Hold me tight! Influence of discriminative features on deep network boundaries [63.627760598441796]
  We propose a new perspective that relates dataset features to the distance of samples to the decision boundary. This enables us to carefully tweak the position of the training samples and measure the induced changes on the boundaries of CNNs trained on large-scale vision datasets.
  arXiv  Detail & Related papers  (2020-02-15T09:29:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.