Interpreting Multivariate Shapley Interactions in DNNs

• URL: http://arxiv.org/abs/2010.05045v4
• Date: Wed, 3 Feb 2021 09:12:47 GMT
• Title: Interpreting Multivariate Shapley Interactions in DNNs
• Authors: Hao Zhang, Yichen Xie, Longjie Zheng, Die Zhang, Quanshi Zhang
• Abstract summary: This paper aims to explain deep neural networks (DNNs) from the perspective of multivariate interactions. In this paper, we define and quantify the significance of interactions among multiple input variables of the DNN.
• Score: 33.67263820904767
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper aims to explain deep neural networks (DNNs) from the perspective of multivariate interactions. In this paper, we define and quantify the significance of interactions among multiple input variables of the DNN. Input variables with strong interactions usually form a coalition and reflect prototype features, which are memorized and used by the DNN for inference. We define the significance of interactions based on the Shapley value, which is designed to assign the attribution value of each input variable to the inference. We have conducted experiments with various DNNs. Experimental results have demonstrated the effectiveness of the proposed method.

Related papers

• Towards the Dynamics of a DNN Learning Symbolic Interactions [20.493304123269446]
  In recent years, a series of theorems have been proven to show that given an input sample, a small number of interactions between input variables can be considered as primitive inference patterns. This study proves the two-phase dynamics of a deep neural network (DNN) learning interactions.
  arXiv  Detail & Related papers  (2024-07-27T07:34:49Z)
• Two-Phase Dynamics of Interactions Explains the Starting Point of a DNN Learning Over-Fitted Features [68.3512123520931]
  We investigate the dynamics of a deep neural network (DNN) learning interactions. In this paper, we discover the DNN learns interactions in two phases. The first phase mainly penalizes interactions of medium and high orders, and the second phase mainly learns interactions of gradually increasing orders.
  arXiv  Detail & Related papers  (2024-05-16T17:13:25Z)
• Defining and Extracting generalizable interaction primitives from DNNs [22.79131582164054]
  We develop a new method to extract interactions that are shared by different deep neural networks (DNNs) Experiments show that the extracted interactions can better reflect common knowledge shared by different DNNs.
  arXiv  Detail & Related papers  (2024-01-29T17:21:41Z)
• Deep Neural Networks Tend To Extrapolate Predictably [51.303814412294514]
  neural network predictions tend to be unpredictable and overconfident when faced with out-of-distribution (OOD) inputs. We observe that neural network predictions often tend towards a constant value as input data becomes increasingly OOD. We show how one can leverage our insights in practice to enable risk-sensitive decision-making in the presence of OOD inputs.
  arXiv  Detail & Related papers  (2023-10-02T03:25:32Z)
• Technical Note: Defining and Quantifying AND-OR Interactions for Faithful and Concise Explanation of DNNs [24.099892982101398]
  We aim to explain a deep neural network (DNN) by quantifying the encoded interactions between input variables. Specifically, we first rethink the definition of interactions, and then formally define faithfulness and conciseness for interaction-based explanation.
  arXiv  Detail & Related papers  (2023-04-26T06:33:31Z)
• Discovering and Explaining the Representation Bottleneck of DNNs [21.121270460158712]
  This paper explores the bottleneck of feature representations of deep neural networks (DNNs) We focus on the multi-order interaction between input variables, where the order represents the complexity of interactions. We discover that a DNN is more likely to encode both too simple interactions and too complex interactions, but usually fails to learn interactions of intermediate complexity.
  arXiv  Detail & Related papers  (2021-11-11T14:35:20Z)
• Discrete-Valued Neural Communication [85.3675647398994]
  We show that restricting the transmitted information among components to discrete representations is a beneficial bottleneck. Even though individuals have different understandings of what a "cat" is based on their specific experiences, the shared discrete token makes it possible for communication among individuals to be unimpeded by individual differences in internal representation. We extend the quantization mechanism from the Vector-Quantized Variational Autoencoder to multi-headed discretization with shared codebooks and use it for discrete-valued neural communication.
  arXiv  Detail & Related papers  (2021-07-06T03:09:25Z)
• Towards Interaction Detection Using Topological Analysis on Neural Networks [55.74562391439507]
  In neural networks, any interacting features must follow a strongly weighted connection to common hidden units. We propose a new measure for quantifying interaction strength, based upon the well-received theory of persistent homology. A Persistence Interaction detection(PID) algorithm is developed to efficiently detect interactions.
  arXiv  Detail & Related papers  (2020-10-25T02:15:24Z)
• Building Interpretable Interaction Trees for Deep NLP Models [38.50154540331266]
  Six metrics are proposed to analyze properties of interactions between constituents in a sentence. Our method is used to quantify word interactions encoded inside the BERT, ELMo, LSTM, CNN, and Transformer networks.
  arXiv  Detail & Related papers  (2020-06-29T10:26:50Z)
• DEPARA: Deep Attribution Graph for Deep Knowledge Transferability [91.06106524522237]
  We propose the DEeP Attribution gRAph (DEPARA) to investigate the transferability of knowledge learned from PR-DNNs. In DEPARA, nodes correspond to the inputs and are represented by their vectorized attribution maps with regards to the outputs of the PR-DNN. The knowledge transferability of two PR-DNNs is measured by the similarity of their corresponding DEPARAs.
  arXiv  Detail & Related papers  (2020-03-17T02:07:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.