Fine-Grained Neural Network Explanation by Identifying Input Features with Predictive Information

• URL: http://arxiv.org/abs/2110.01471v1
• Date: Mon, 4 Oct 2021 14:13:42 GMT
• Title: Fine-Grained Neural Network Explanation by Identifying Input Features with Predictive Information
• Authors: Yang Zhang, Ashkan Khakzar, Yawei Li, Azade Farshad, Seong Tae Kim, Nassir Navab
• Abstract summary: We propose a method to identify features with predictive information in the input domain. The core idea of our method is leveraging a bottleneck on the input that only lets input features associated with predictive latent features pass through.
• Score: 53.28701922632817
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: One principal approach for illuminating a black-box neural network is feature attribution, i.e. identifying the importance of input features for the network's prediction. The predictive information of features is recently proposed as a proxy for the measure of their importance. So far, the predictive information is only identified for latent features by placing an information bottleneck within the network. We propose a method to identify features with predictive information in the input domain. The method results in fine-grained identification of input features' information and is agnostic to network architecture. The core idea of our method is leveraging a bottleneck on the input that only lets input features associated with predictive latent features pass through. We compare our method with several feature attribution methods using mainstream feature attribution evaluation experiments. The code is publicly available.

Related papers

• Feature Selection for Network Intrusion Detection [3.7414804164475983]
  We present a novel information-theoretic method that facilitates the exclusion of non-informative features when detecting network intrusions. The proposed method is based on function approximation using a neural network, which enables a version of our approach that incorporates a recurrent layer.
  arXiv  Detail & Related papers  (2024-11-18T14:25:55Z)
• Automatic Input Feature Relevance via Spectral Neural Networks [0.9236074230806581]
  We propose a novel method to estimate the relative importance of the input components for a Deep Neural Network. This is achieved by leveraging on a spectral re-parametrization of the optimization process. The technique is successfully challenged against both synthetic and real data.
  arXiv  Detail & Related papers  (2024-06-03T10:39:12Z)
• Tractable Function-Space Variational Inference in Bayesian Neural Networks [72.97620734290139]
  A popular approach for estimating the predictive uncertainty of neural networks is to define a prior distribution over the network parameters. We propose a scalable function-space variational inference method that allows incorporating prior information. We show that the proposed method leads to state-of-the-art uncertainty estimation and predictive performance on a range of prediction tasks.
  arXiv  Detail & Related papers  (2023-12-28T18:33:26Z)
• Provable Data Subset Selection For Efficient Neural Network Training [73.34254513162898]
  We introduce the first algorithm to construct coresets for emphRBFNNs, i.e., small weighted subsets that approximate the loss of the input data on any radial basis function network. We then perform empirical evaluations on function approximation and dataset subset selection on popular network architectures and data sets.
  arXiv  Detail & Related papers  (2023-03-09T10:08:34Z)
• Representation Learning for Sequence Data with Deep Autoencoding Predictive Components [96.42805872177067]
  We propose a self-supervised representation learning method for sequence data, based on the intuition that useful representations of sequence data should exhibit a simple structure in the latent space. We encourage this latent structure by maximizing an estimate of predictive information of latent feature sequences, which is the mutual information between past and future windows at each time step. We demonstrate that our method recovers the latent space of noisy dynamical systems, extracts predictive features for forecasting tasks, and improves automatic speech recognition when used to pretrain the encoder on large amounts of unlabeled data.
  arXiv  Detail & Related papers  (2020-10-07T03:34:01Z)
• From Handcrafted to Deep Features for Pedestrian Detection: A Survey [148.35460817092908]
  Pedestrian detection is an important but challenging problem in computer vision. Over the past decade, significant improvement has been witnessed with the help of handcrafted features and deep features. In addition to single-spectral pedestrian detection, we also review multi-spectral pedestrian detection.
  arXiv  Detail & Related papers  (2020-10-01T14:51:10Z)
• Counterfactual Explanation Based on Gradual Construction for Deep Networks [17.79934085808291]
  The patterns that deep networks have learned from a training dataset can be grasped by observing the feature variation among various classes. Current approaches perform the feature modification to increase the classification probability for the target class irrespective of the internal characteristics of deep networks. We propose a counterfactual explanation method that exploits the statistics learned from a training dataset.
  arXiv  Detail & Related papers  (2020-08-05T01:18:31Z)
• Detecting unusual input to neural networks [0.48733623015338234]
  We study a method that judges the unusualness of an input by evaluating its informative content compared to the learned parameters. This technique can be used to judge whether a network is suitable for processing a certain input and to raise a red flag that unexpected behavior might lie ahead.
  arXiv  Detail & Related papers  (2020-06-15T10:48:43Z)
• Learning to Ask Medical Questions using Reinforcement Learning [9.376814468000955]
  A reinforcement learning agent iteratively selects certain features to be unmasked, and uses them to predict an outcome when it is sufficiently confident. A key component of our approach is a guesser network, trained to predict the outcome from the selected features and parametrizing the reward function. Applying our method to a national survey dataset, we show that it not only outperforms strong baselines when requiring the prediction to be made based on a small number of input features, but is also highly more interpretable.
  arXiv  Detail & Related papers  (2020-03-31T18:21:46Z)
• Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations [143.3053365553897]
  We describe a procedure for removing dependency on a cohort of training data from a trained deep network. We introduce a new bound on how much information can be extracted per query about the forgotten cohort. We exploit the connections between the activation and weight dynamics of a DNN inspired by Neural Tangent Kernels to compute the information in the activations.
  arXiv  Detail & Related papers  (2020-03-05T23:17:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.