On Model Explanations with Transferable Neural Pathways

• URL: http://arxiv.org/abs/2309.09887v1
• Date: Mon, 18 Sep 2023 15:50:38 GMT
• Title: On Model Explanations with Transferable Neural Pathways
• Authors: Xinmiao Lin, Wentao Bao, Qi Yu, Yu Kong
• Abstract summary: We propose a Generative Class-relevant Neural Pathway (GEN-CNP) model that learns to predict the neural pathways from the target model's feature maps. We propose to transfer the class-relevant neural pathways to explain samples of the same class and show experimentally and qualitatively their faithfulness and interpretability.
• Score: 41.2093021477798
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural pathways as model explanations consist of a sparse set of neurons that provide the same level of prediction performance as the whole model. Existing methods primarily focus on accuracy and sparsity but the generated pathways may offer limited interpretability thus fall short in explaining the model behavior. In this paper, we suggest two interpretability criteria of neural pathways: (i) same-class neural pathways should primarily consist of class-relevant neurons; (ii) each instance's neural pathway sparsity should be optimally determined. To this end, we propose a Generative Class-relevant Neural Pathway (GEN-CNP) model that learns to predict the neural pathways from the target model's feature maps. We propose to learn class-relevant information from features of deep and shallow layers such that same-class neural pathways exhibit high similarity. We further impose a faithfulness criterion for GEN-CNP to generate pathways with instance-specific sparsity. We propose to transfer the class-relevant neural pathways to explain samples of the same class and show experimentally and qualitatively their faithfulness and interpretability.

Related papers

• Novel Kernel Models and Exact Representor Theory for Neural Networks Beyond the Over-Parameterized Regime [52.00917519626559]
  This paper presents two models of neural-networks and their training applicable to neural networks of arbitrary width, depth and topology. We also present an exact novel representor theory for layer-wise neural network training with unregularized gradient descent in terms of a local-extrinsic neural kernel (LeNK) This representor theory gives insight into the role of higher-order statistics in neural network training and the effect of kernel evolution in neural-network kernel models.
  arXiv  Detail & Related papers  (2024-05-24T06:30:36Z)
• Towards Generating Informative Textual Description for Neurons in Language Models [6.884227665279812]
  We propose a framework that ties textual descriptions to neurons. In particular, our experiment shows that the proposed approach achieves 75% precision@2, and 50% recall@2
  arXiv  Detail & Related papers  (2024-01-30T04:06:25Z)
• A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models [13.283281356356161]
  We review the literature on statistical theories of neural networks from three perspectives. Results on excess risks for neural networks are reviewed. Papers that attempt to answer how the neural network finds the solution that can generalize well on unseen data'' are reviewed.
  arXiv  Detail & Related papers  (2024-01-14T02:30:19Z)
• Feature visualization for convolutional neural network models trained on neuroimaging data [0.0]
  We show for the first time results using feature visualization of convolutional neural networks (CNNs) We have trained CNNs for different tasks including sex classification and artificial lesion classification based on structural magnetic resonance imaging (MRI) data. The resulting images reveal the learned concepts of the artificial lesions, including their shapes, but remain hard to interpret for abstract features in the sex classification task.
  arXiv  Detail & Related papers  (2022-03-24T15:24:38Z)
• EINNs: Epidemiologically-Informed Neural Networks [75.34199997857341]
  We introduce a new class of physics-informed neural networks-EINN-crafted for epidemic forecasting. We investigate how to leverage both the theoretical flexibility provided by mechanistic models as well as the data-driven expressability afforded by AI models.
  arXiv  Detail & Related papers  (2022-02-21T18:59:03Z)
• The Causal Neural Connection: Expressiveness, Learnability, and Inference [125.57815987218756]
  An object called structural causal model (SCM) represents a collection of mechanisms and sources of random variation of the system under investigation. In this paper, we show that the causal hierarchy theorem (Thm. 1, Bareinboim et al., 2020) still holds for neural models. We introduce a special type of SCM called a neural causal model (NCM), and formalize a new type of inductive bias to encode structural constraints necessary for performing causal inferences.
  arXiv  Detail & Related papers  (2021-07-02T01:55:18Z)
• The Neural Coding Framework for Learning Generative Models [91.0357317238509]
  We propose a novel neural generative model inspired by the theory of predictive processing in the brain. In a similar way, artificial neurons in our generative model predict what neighboring neurons will do, and adjust their parameters based on how well the predictions matched reality.
  arXiv  Detail & Related papers  (2020-12-07T01:20:38Z)
• A Bayesian Perspective on Training Speed and Model Selection [51.15664724311443]
  We show that a measure of a model's training speed can be used to estimate its marginal likelihood. We verify our results in model selection tasks for linear models and for the infinite-width limit of deep neural networks. Our results suggest a promising new direction towards explaining why neural networks trained with gradient descent are biased towards functions that generalize well.
  arXiv  Detail & Related papers  (2020-10-27T17:56:14Z)
• Spiking Neural Networks -- Part II: Detecting Spatio-Temporal Patterns [38.518936229794214]
  Spiking Neural Networks (SNNs) have the unique ability to detect information in encoded-temporal signals. We review models and training algorithms for the dominant approach that considers SNNs as a Recurrent Neural Network (RNN) We describe an alternative approach that relies on probabilistic models for spiking neurons, allowing the derivation of local learning rules via gradient estimates.
  arXiv  Detail & Related papers  (2020-10-27T11:47:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.