Topological Understanding of Neural Networks, a survey

• URL: http://arxiv.org/abs/2301.09742v1
• Date: Mon, 23 Jan 2023 22:11:37 GMT
• Title: Topological Understanding of Neural Networks, a survey
• Authors: Tushar Pandey
• Abstract summary: We look at the internal structure of neural networks which is usually treated as a black box. We review the significance of different activation functions, types of network architectures associated to them, and some empirical data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We look at the internal structure of neural networks which is usually treated as a black box. The easiest and the most comprehensible thing to do is to look at a binary classification and try to understand the approach a neural network takes. We review the significance of different activation functions, types of network architectures associated to them, and some empirical data. We find some interesting observations and a possibility to build upon the ideas to verify the process for real datasets. We suggest some possible experiments to look forward to in three different directions.

Related papers

• Deception Detection from Linguistic and Physiological Data Streams Using Bimodal Convolutional Neural Networks [19.639533220155965]
  This paper explores the application of convolutional neural networks for the purpose of multimodal deception detection. We use a dataset built by interviewing 104 subjects about two topics, with one truthful and one falsified response from each subject about each topic.
  arXiv  Detail & Related papers  (2023-11-18T02:44:33Z)
• Formal Conceptual Views in Neural Networks [0.0]
  We introduce two notions for conceptual views of a neural network, specifically a many-valued and a symbolic view. We test the conceptual expressivity of our novel views through different experiments on the ImageNet and Fruit-360 data sets. We demonstrate how conceptual views can be applied for abductive learning of human comprehensible rules from neurons.
  arXiv  Detail & Related papers  (2022-09-27T16:38:24Z)
• Information Flow in Deep Neural Networks [0.6922389632860545]
  There is no comprehensive theoretical understanding of how deep neural networks work or are structured. Deep networks are often seen as black boxes with unclear interpretations and reliability. This work aims to apply principles and techniques from information theory to deep learning models to increase our theoretical understanding and design better algorithms.
  arXiv  Detail & Related papers  (2022-02-10T23:32:26Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)
• A neural anisotropic view of underspecification in deep learning [60.119023683371736]
  We show that the way neural networks handle the underspecification of problems is highly dependent on the data representation. Our results highlight that understanding the architectural inductive bias in deep learning is fundamental to address the fairness, robustness, and generalization of these systems.
  arXiv  Detail & Related papers  (2021-04-29T14:31:09Z)
• Analyzing Representations inside Convolutional Neural Networks [8.803054559188048]
  We propose a framework to categorize the concepts a network learns based on the way it clusters a set of input examples. This framework is unsupervised and can work without any labels for input features. We extensively evaluate the proposed method and demonstrate that it produces human-understandable and coherent concepts.
  arXiv  Detail & Related papers  (2020-12-23T07:10:17Z)
• Understanding the Role of Individual Units in a Deep Neural Network [85.23117441162772]
  We present an analytic framework to systematically identify hidden units within image classification and image generation networks. First, we analyze a convolutional neural network (CNN) trained on scene classification and discover units that match a diverse set of object concepts. Second, we use a similar analytic method to analyze a generative adversarial network (GAN) model trained to generate scenes.
  arXiv  Detail & Related papers  (2020-09-10T17:59:10Z)
• The Representation Theory of Neural Networks [7.724617675868718]
  We show that neural networks can be represented via the mathematical theory of quiver representations. We show that network quivers gently adapt to common neural network concepts. We also provide a quiver representation model to understand how a neural network creates representations from the data.
  arXiv  Detail & Related papers  (2020-07-23T19:02:14Z)
• Graph Structure of Neural Networks [104.33754950606298]
  We show how the graph structure of neural networks affect their predictive performance. A "sweet spot" of relational graphs leads to neural networks with significantly improved predictive performance. Top-performing neural networks have graph structure surprisingly similar to those of real biological neural networks.
  arXiv  Detail & Related papers  (2020-07-13T17:59:31Z)
• Towards Understanding Hierarchical Learning: Benefits of Neural Representations [160.33479656108926]
  In this work, we demonstrate that intermediate neural representations add more flexibility to neural networks. We show that neural representation can achieve improved sample complexities compared with the raw input. Our results characterize when neural representations are beneficial, and may provide a new perspective on why depth is important in deep learning.
  arXiv  Detail & Related papers  (2020-06-24T02:44:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.