Sparse Oblique Decision Trees: A Tool to Understand and Manipulate Neural Net Features

• URL: http://arxiv.org/abs/2104.02922v1
• Date: Wed, 7 Apr 2021 05:31:08 GMT
• Title: Sparse Oblique Decision Trees: A Tool to Understand and Manipulate Neural Net Features
• Authors: Suryabhan Singh Hada and Miguel \'A. Carreira-Perpi\~n\'an and Arman Zharmagambetov
• Abstract summary: We focus on understanding which of the internal features computed by the neural net are responsible for a particular class. We show we can easily manipulate the neural net features in order to make the net predict, or not predict, a given class, thus showing that it is possible to carry out adversarial attacks at the level of the features.
• Score: 3.222802562733787
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The widespread deployment of deep nets in practical applications has lead to a growing desire to understand how and why such black-box methods perform prediction. Much work has focused on understanding what part of the input pattern (an image, say) is responsible for a particular class being predicted, and how the input may be manipulated to predict a different class. We focus instead on understanding which of the internal features computed by the neural net are responsible for a particular class. We achieve this by mimicking part of the neural net with an oblique decision tree having sparse weight vectors at the decision nodes. Using the recently proposed Tree Alternating Optimization (TAO) algorithm, we are able to learn trees that are both highly accurate and interpretable. Such trees can faithfully mimic the part of the neural net they replaced, and hence they can provide insights into the deep net black box. Further, we show we can easily manipulate the neural net features in order to make the net predict, or not predict, a given class, thus showing that it is possible to carry out adversarial attacks at the level of the features. These insights and manipulations apply globally to the entire training and test set, not just at a local (single-instance) level. We demonstrate this robustly in the MNIST and ImageNet datasets with LeNet5 and VGG networks.

Related papers

• Coding schemes in neural networks learning classification tasks [52.22978725954347]
  We investigate fully-connected, wide neural networks learning classification tasks. We show that the networks acquire strong, data-dependent features. Surprisingly, the nature of the internal representations depends crucially on the neuronal nonlinearity.
  arXiv  Detail & Related papers  (2024-06-24T14:50:05Z)
• Knowledge Distillation of Convolutional Neural Networks through Feature Map Transformation using Decision Trees [2.06682776181122]
  We propose a distillation approach by extracting features from the final layer of the convolutional neural network (CNN) The extracted features are used to train a decision tree to achieve the best accuracy under constraints of depth and nodes. The results encourage interpreting decisions made by the CNNs using decision trees.
  arXiv  Detail & Related papers  (2024-03-10T04:20:51Z)
• Neural Networks are Decision Trees [0.0]
  We show that any neural network having piece-wise linear activation functions can be represented as a decision tree. The representation is equivalence and not an approximation, thus keeping the accuracy of the neural network exactly as is.
  arXiv  Detail & Related papers  (2022-10-11T06:49:51Z)
• Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics [85.31710759801705]
  Current practice requires expensive computational costs in model training for performance prediction. We propose a novel framework for neural network selection by analyzing the governing dynamics over synaptic connections (edges) during training. Our framework is built on the fact that back-propagation during neural network training is equivalent to the dynamical evolution of synaptic connections.
  arXiv  Detail & Related papers  (2022-01-11T20:53:15Z)
• Detecting Modularity in Deep Neural Networks [8.967870619902211]
  We consider the problem of assessing the modularity exhibited by a partitioning of a network's neurons. We propose two proxies for this: importance, which reflects how crucial sets of neurons are to network performance; and coherence, which reflects how consistently their neurons associate with features of the inputs. We show that these partitionings, even ones based only on weights, reveal groups of neurons that are important and coherent.
  arXiv  Detail & Related papers  (2021-10-13T20:33:30Z)
• Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks [79.74580058178594]
  We analyze the performance of training a pruned neural network by analyzing the geometric structure of the objective function. We show that the convex region near a desirable model with guaranteed generalization enlarges as the neural network model is pruned.
  arXiv  Detail & Related papers  (2021-10-12T01:11:07Z)
• Leveraging Sparse Linear Layers for Debuggable Deep Networks [86.94586860037049]
  We show how fitting sparse linear models over learned deep feature representations can lead to more debuggable neural networks. The resulting sparse explanations can help to identify spurious correlations, explain misclassifications, and diagnose model biases in vision and language tasks.
  arXiv  Detail & Related papers  (2021-05-11T08:15:25Z)
• The Connection Between Approximation, Depth Separation and Learnability in Neural Networks [70.55686685872008]
  We study the connection between learnability and approximation capacity. We show that learnability with deep networks of a target function depends on the ability of simpler classes to approximate the target.
  arXiv  Detail & Related papers  (2021-01-31T11:32:30Z)
• Locality Guided Neural Networks for Explainable Artificial Intelligence [12.435539489388708]
  We propose a novel algorithm for back propagation, called Locality Guided Neural Network(LGNN) LGNN preserves locality between neighbouring neurons within each layer of a deep network. In our experiments, we train various VGG and Wide ResNet (WRN) networks for image classification on CIFAR100.
  arXiv  Detail & Related papers  (2020-07-12T23:45:51Z)
• Towards Understanding Hierarchical Learning: Benefits of Neural Representations [160.33479656108926]
  In this work, we demonstrate that intermediate neural representations add more flexibility to neural networks. We show that neural representation can achieve improved sample complexities compared with the raw input. Our results characterize when neural representations are beneficial, and may provide a new perspective on why depth is important in deep learning.
  arXiv  Detail & Related papers  (2020-06-24T02:44:54Z)
• DeepStreamCE: A Streaming Approach to Concept Evolution Detection in Deep Neural Networks [1.933681537640272]
  DeepStreamCE uses streaming approaches for real-time concept evolution detection in deep neural networks. We evaluate DeepStreamCE by training VGG16 convolutional neural networks on combinations of data from the CIFAR-10 dataset. For comparison, we apply the data and VGG16 networks to an open-set deep network solution - OpenMax.
  arXiv  Detail & Related papers  (2020-04-08T16:53:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.