Knowledge Distillation of Convolutional Neural Networks through Feature Map Transformation using Decision Trees

• URL: http://arxiv.org/abs/2403.06089v1
• Date: Sun, 10 Mar 2024 04:20:51 GMT
• Title: Knowledge Distillation of Convolutional Neural Networks through Feature Map Transformation using Decision Trees
• Authors: Maddimsetti Srinivas and Debdoot Sheet
• Abstract summary: We propose a distillation approach by extracting features from the final layer of the convolutional neural network (CNN) The extracted features are used to train a decision tree to achieve the best accuracy under constraints of depth and nodes. The results encourage interpreting decisions made by the CNNs using decision trees.
• Score: 2.06682776181122
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The interpretation of reasoning by Deep Neural Networks (DNN) is still challenging due to their perceived black-box nature. Therefore, deploying DNNs in several real-world tasks is restricted by the lack of transparency of these models. We propose a distillation approach by extracting features from the final layer of the convolutional neural network (CNN) to address insights to its reasoning. The feature maps in the final layer of a CNN are transformed into a one-dimensional feature vector using a fully connected layer. Subsequently, the extracted features are used to train a decision tree to achieve the best accuracy under constraints of depth and nodes. We use the medical images of dermaMNIST, octMNIST, and pneumoniaMNIST from the medical MNIST datasets to demonstrate our proposed work. We observed that performance of the decision tree is as good as a CNN with minimum complexity. The results encourage interpreting decisions made by the CNNs using decision trees.

Related papers

• CNN2GNN: How to Bridge CNN with GNN [59.42117676779735]
  We propose a novel CNN2GNN framework to unify CNN and GNN together via distillation. The performance of distilled boosted'' two-layer GNN on Mini-ImageNet is much higher than CNN containing dozens of layers such as ResNet152.
  arXiv  Detail & Related papers  (2024-04-23T08:19:08Z)
• Deeply Explain CNN via Hierarchical Decomposition [75.01251659472584]
  In computer vision, some attribution methods for explaining CNNs attempt to study how the intermediate features affect the network prediction. This paper introduces a hierarchical decomposition framework to explain CNN's decision-making process in a top-down manner.
  arXiv  Detail & Related papers  (2022-01-23T07:56:04Z)
• Visual Explanations for Convolutional Neural Networks via Latent Traversal of Generative Adversarial Networks [17.475341881835355]
  We present a method for interpreting what a convolutional neural network (CNN) has learned by utilizing Generative Adversarial Networks (GANs) Our GAN framework disentangles lung structure from COVID-19 features. Using this GAN, we can visualize the transition of a pair of COVID negative lungs in a chest radiograph to a COVID positive pair by interpolating in the latent space of the GAN.
  arXiv  Detail & Related papers  (2021-10-29T23:26:09Z)
• Convolutional Neural Networks Demystified: A Matched Filtering Perspective Based Tutorial [7.826806223782053]
  Convolutional Neural Networks (CNN) are a de-facto standard for the analysis of large volumes of signals and images. We revisit their operation from first principles and a matched filtering perspective. It is our hope that this tutorial will help shed new light and physical intuition into the understanding and further development of deep neural networks.
  arXiv  Detail & Related papers  (2021-08-26T09:07:49Z)
• Topological Measurement of Deep Neural Networks Using Persistent Homology [0.7919213739992464]
  The inner representation of deep neural networks (DNNs) is indecipherable. Persistent homology (PH) was employed for investigating the complexities of trained DNNs.
  arXiv  Detail & Related papers  (2021-06-06T03:06:15Z)
• BreakingBED -- Breaking Binary and Efficient Deep Neural Networks by Adversarial Attacks [65.2021953284622]
  We study robustness of CNNs against white-box and black-box adversarial attacks. Results are shown for distilled CNNs, agent-based state-of-the-art pruned models, and binarized neural networks.
  arXiv  Detail & Related papers  (2021-03-14T20:43:19Z)
• The Mind's Eye: Visualizing Class-Agnostic Features of CNNs [92.39082696657874]
  We propose an approach to visually interpret CNN features given a set of images by creating corresponding images that depict the most informative features of a specific layer. Our method uses a dual-objective activation and distance loss, without requiring a generator network nor modifications to the original model.
  arXiv  Detail & Related papers  (2021-01-29T07:46:39Z)
• Simplifying the explanation of deep neural networks with sufficient and necessary feature-sets: case of text classification [0.0]
  Deep neural networks (DNN) have demonstrated impressive performances solving a wide range of problems in various domains such as medicine, finance, law, etc. Despite their great performances, they have long been considered as black-box systems, providing good results without being able to explain them. This article proposes a method to simplify the prediction explanation of One-Dimensional (1D) Convolutional Neural Networks (CNN) by identifying sufficient and necessary features-sets.
  arXiv  Detail & Related papers  (2020-10-08T02:01:21Z)
• Boosting Deep Neural Networks with Geometrical Prior Knowledge: A Survey [77.99182201815763]
  Deep Neural Networks (DNNs) achieve state-of-the-art results in many different problem settings. DNNs are often treated as black box systems, which complicates their evaluation and validation. One promising field, inspired by the success of convolutional neural networks (CNNs) in computer vision tasks, is to incorporate knowledge about symmetric geometrical transformations.
  arXiv  Detail & Related papers  (2020-06-30T14:56:05Z)
• Hold me tight! Influence of discriminative features on deep network boundaries [63.627760598441796]
  We propose a new perspective that relates dataset features to the distance of samples to the decision boundary. This enables us to carefully tweak the position of the training samples and measure the induced changes on the boundaries of CNNs trained on large-scale vision datasets.
  arXiv  Detail & Related papers  (2020-02-15T09:29:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.