Analyzing Representations inside Convolutional Neural Networks
- URL: http://arxiv.org/abs/2012.12516v1
- Date: Wed, 23 Dec 2020 07:10:17 GMT
- Title: Analyzing Representations inside Convolutional Neural Networks
- Authors: Uday Singh Saini, Evangelos E. Papalexakis
- Abstract summary: We propose a framework to categorize the concepts a network learns based on the way it clusters a set of input examples.
This framework is unsupervised and can work without any labels for input features.
We extensively evaluate the proposed method and demonstrate that it produces human-understandable and coherent concepts.
- Score: 8.803054559188048
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: How can we discover and succinctly summarize the concepts that a neural
network has learned? Such a task is of great importance in applications of
networks in areas of inference that involve classification, like medical
diagnosis based on fMRI/x-ray etc. In this work, we propose a framework to
categorize the concepts a network learns based on the way it clusters a set of
input examples, clusters neurons based on the examples they activate for, and
input features all in the same latent space. This framework is unsupervised and
can work without any labels for input features, it only needs access to
internal activations of the network for each input example, thereby making it
widely applicable. We extensively evaluate the proposed method and demonstrate
that it produces human-understandable and coherent concepts that a ResNet-18
has learned on the CIFAR-100 dataset.
Related papers
- Coding schemes in neural networks learning classification tasks [52.22978725954347]
We investigate fully-connected, wide neural networks learning classification tasks.
We show that the networks acquire strong, data-dependent features.
Surprisingly, the nature of the internal representations depends crucially on the neuronal nonlinearity.
arXiv Detail & Related papers (2024-06-24T14:50:05Z) - Conditional computation in neural networks: principles and research trends [48.14569369912931]
This article summarizes principles and ideas from the emerging area of applying textitconditional computation methods to the design of neural networks.
In particular, we focus on neural networks that can dynamically activate or de-activate parts of their computational graph conditionally on their input.
arXiv Detail & Related papers (2024-03-12T11:56:38Z) - DISCOVER: Making Vision Networks Interpretable via Competition and
Dissection [11.028520416752325]
This work contributes to post-hoc interpretability, and specifically Network Dissection.
Our goal is to present a framework that makes it easier to discover the individual functionality of each neuron in a network trained on a vision task.
arXiv Detail & Related papers (2023-10-07T21:57:23Z) - Topological Understanding of Neural Networks, a survey [0.0]
We look at the internal structure of neural networks which is usually treated as a black box.
We review the significance of different activation functions, types of network architectures associated to them, and some empirical data.
arXiv Detail & Related papers (2023-01-23T22:11:37Z) - Neural Activation Patterns (NAPs): Visual Explainability of Learned
Concepts [8.562628320010035]
We present a method that takes into account the entire activation distribution.
By extracting similar activation profiles within the high-dimensional activation space of a neural network layer, we find groups of inputs that are treated similarly.
These input groups represent neural activation patterns (NAPs) and can be used to visualize and interpret learned layer concepts.
arXiv Detail & Related papers (2022-06-20T09:05:57Z) - Quasi-orthogonality and intrinsic dimensions as measures of learning and
generalisation [55.80128181112308]
We show that dimensionality and quasi-orthogonality of neural networks' feature space may jointly serve as network's performance discriminants.
Our findings suggest important relationships between the networks' final performance and properties of their randomly initialised feature spaces.
arXiv Detail & Related papers (2022-03-30T21:47:32Z) - Pointer Value Retrieval: A new benchmark for understanding the limits of
neural network generalization [40.21297628440919]
We introduce a novel benchmark, Pointer Value Retrieval (PVR) tasks, that explore the limits of neural network generalization.
PVR tasks can consist of visual as well as symbolic inputs, each with varying levels of difficulty.
We demonstrate that this task structure provides a rich testbed for understanding generalization.
arXiv Detail & Related papers (2021-07-27T03:50:31Z) - Joint Learning of Neural Transfer and Architecture Adaptation for Image
Recognition [77.95361323613147]
Current state-of-the-art visual recognition systems rely on pretraining a neural network on a large-scale dataset and finetuning the network weights on a smaller dataset.
In this work, we prove that dynamically adapting network architectures tailored for each domain task along with weight finetuning benefits in both efficiency and effectiveness.
Our method can be easily generalized to an unsupervised paradigm by replacing supernet training with self-supervised learning in the source domain tasks and performing linear evaluation in the downstream tasks.
arXiv Detail & Related papers (2021-03-31T08:15:17Z) - Quantitative Effectiveness Assessment and Role Categorization of
Individual Units in Convolutional Neural Networks [23.965084518584298]
We propose a method for quantitatively clarifying the status and usefulness of single unit of CNN in image classification tasks.
The technical substance of our method is ranking the importance of unit for each class in classification based on calculation of specifically defined entropy.
All of the network units are divided into four categories according to their performance on training and testing data.
arXiv Detail & Related papers (2021-03-17T15:18:18Z) - Understanding the Role of Individual Units in a Deep Neural Network [85.23117441162772]
We present an analytic framework to systematically identify hidden units within image classification and image generation networks.
First, we analyze a convolutional neural network (CNN) trained on scene classification and discover units that match a diverse set of object concepts.
Second, we use a similar analytic method to analyze a generative adversarial network (GAN) model trained to generate scenes.
arXiv Detail & Related papers (2020-09-10T17:59:10Z) - Graph Prototypical Networks for Few-shot Learning on Attributed Networks [72.31180045017835]
We propose a graph meta-learning framework -- Graph Prototypical Networks (GPN)
GPN is able to perform textitmeta-learning on an attributed network and derive a highly generalizable model for handling the target classification task.
arXiv Detail & Related papers (2020-06-23T04:13:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.