Topological Data Analysis of Neural Network Layer Representations

• URL: http://arxiv.org/abs/2208.06438v1
• Date: Fri, 1 Jul 2022 00:51:19 GMT
• Title: Topological Data Analysis of Neural Network Layer Representations
• Authors: Archie Shahidullah
• Abstract summary: topological features of a simple feedforward neural network's layer representations of a modified torus with a Klein bottle-like twist were computed. The resulting noise hampered the ability of persistent homology to compute these features.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper is a cursory study on how topological features are preserved within the internal representations of neural network layers. Using techniques from topological data analysis, namely persistent homology, the topological features of a simple feedforward neural network's layer representations of a modified torus with a Klein bottle-like twist were computed. The network appeared to approximate homeomorphisms in early layers, before significantly changing the topology of the data in deeper layers. The resulting noise hampered the ability of persistent homology to compute these features, however similar topological features seemed to persist longer in a network with a bijective activation function.

Related papers

• A rank decomposition for the topological classification of neural representations [0.0]
  In this work, we leverage the fact that neural networks are equivalent to continuous piecewise-affine maps. We study the homology groups of the quotient of a manifold $mathcalM$ and a subset $A$, assuming some minimal properties on these spaces. We show that in randomly narrow networks, there will be regions in which the (co)homology groups of a data manifold can change.
  arXiv  Detail & Related papers  (2024-04-30T17:01:20Z)
• Topological Expressivity of ReLU Neural Networks [0.0]
  We study the expressivity of ReLU neural networks in the setting of a binary classification problem from a topological perspective. Results show that deep ReLU neural networks are exponentially more powerful than shallow ones in terms of topological simplification.
  arXiv  Detail & Related papers  (2023-10-17T10:28:00Z)
• Addressing caveats of neural persistence with deep graph persistence [54.424983583720675]
  We find that the variance of network weights and spatial concentration of large weights are the main factors that impact neural persistence. We propose an extension of the filtration underlying neural persistence to the whole neural network instead of single layers. This yields our deep graph persistence measure, which implicitly incorporates persistent paths through the network and alleviates variance-related issues.
  arXiv  Detail & Related papers  (2023-07-20T13:34:11Z)
• Data Topology-Dependent Upper Bounds of Neural Network Widths [52.58441144171022]
  We first show that a three-layer neural network can be designed to approximate an indicator function over a compact set. This is then extended to a simplicial complex, deriving width upper bounds based on its topological structure. We prove the universal approximation property of three-layer ReLU networks using our topological approach.
  arXiv  Detail & Related papers  (2023-05-25T14:17:15Z)
• Rethinking Persistent Homology for Visual Recognition [27.625893409863295]
  This paper performs a detailed analysis of the effectiveness of topological properties for image classification in various training scenarios. We identify the scenarios that benefit the most from topological features, e.g., training simple networks on small datasets.
  arXiv  Detail & Related papers  (2022-07-09T08:01:11Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Activation Landscapes as a Topological Summary of Neural Network Performance [0.0]
  We study how data transforms as it passes through successive layers of a deep neural network (DNN) We compute the persistent homology of the activation data for each layer of the network and summarize this information using persistence landscapes. The resulting feature map provides both an informative visual- ization of the network and a kernel for statistical analysis and machine learning.
  arXiv  Detail & Related papers  (2021-10-19T17:45:36Z)
• Localized Persistent Homologies for more Effective Deep Learning [60.78456721890412]
  We introduce an approach that relies on a new filtration function to account for location during network training. We demonstrate experimentally on 2D images of roads and 3D image stacks of neuronal processes that networks trained in this manner are better at recovering the topology of the curvilinear structures they extract.
  arXiv  Detail & Related papers  (2021-10-12T19:28:39Z)
• Going beyond p-convolutions to learn grayscale morphological operators [64.38361575778237]
  We present two new morphological layers based on the same principle as the p-convolutional layer. In this work, we present two new morphological layers based on the same principle as the p-convolutional layer.
  arXiv  Detail & Related papers  (2021-02-19T17:22:16Z)
• Topological obstructions in neural networks learning [67.8848058842671]
  We study global properties of the loss gradient function flow. We use topological data analysis of the loss function and its Morse complex to relate local behavior along gradient trajectories with global properties of the loss surface.
  arXiv  Detail & Related papers  (2020-12-31T18:53:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.