A Structural Approach to the Design of Domain Specific Neural Network Architectures

• URL: http://arxiv.org/abs/2301.09381v1
• Date: Mon, 23 Jan 2023 11:50:57 GMT
• Title: A Structural Approach to the Design of Domain Specific Neural Network Architectures
• Authors: Gerrit Nolte
• Abstract summary: This thesis aims to provide a theoretical evaluation of geometric deep learning. It compiles theoretical results that characterize the properties of invariant neural networks with respect to learning performance.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This is a master's thesis concerning the theoretical ideas of geometric deep learning. Geometric deep learning aims to provide a structured characterization of neural network architectures, specifically focused on the ideas of invariance and equivariance of data with respect to given transformations. This thesis aims to provide a theoretical evaluation of geometric deep learning, compiling theoretical results that characterize the properties of invariant neural networks with respect to learning performance.

Related papers

• Structure of Artificial Neural Networks -- Empirical Investigations [0.0]
  Within one decade, Deep Learning overtook the dominating solution methods of countless problems of artificial intelligence. With a formal definition for structures of neural networks, neural architecture search problems and solution methods can be formulated under a common framework. Does structure make a difference or can it be chosen arbitrarily?
  arXiv  Detail & Related papers  (2024-10-12T16:13:28Z)
• Tropical Expressivity of Neural Networks [0.0]
  We use tropical geometry to characterize and study various architectural aspects of neural networks. We present a new algorithm that computes the exact number of their linear regions.
  arXiv  Detail & Related papers  (2024-05-30T15:45:03Z)
• Riemannian Residual Neural Networks [58.925132597945634]
  We show how to extend the residual neural network (ResNet) ResNets have become ubiquitous in machine learning due to their beneficial learning properties, excellent empirical results, and easy-to-incorporate nature when building varied neural networks.
  arXiv  Detail & Related papers  (2023-10-16T02:12:32Z)
• Deep neural networks architectures from the perspective of manifold learning [0.0]
  This paper is a comprehensive comparison and description of neural network architectures in terms of ge-ometry and topology. We focus on the internal representation of neural networks and on the dynamics of changes in the topology and geometry of a data manifold on different layers.
  arXiv  Detail & Related papers  (2023-06-06T04:57:39Z)
• The Neural Race Reduction: Dynamics of Abstraction in Gated Networks [12.130628846129973]
  We introduce the Gated Deep Linear Network framework that schematizes how pathways of information flow impact learning dynamics. We derive an exact reduction and, for certain cases, exact solutions to the dynamics of learning. Our work gives rise to general hypotheses relating neural architecture to learning and provides a mathematical approach towards understanding the design of more complex architectures.
  arXiv  Detail & Related papers  (2022-07-21T12:01:03Z)
• Topology and geometry of data manifold in deep learning [0.0]
  This article describes and substantiates the geometric and topological view of the learning process of neural networks. We present a wide range of experiments on different datasets and different configurations of convolutional neural network architectures. Our work is a contribution to the development of an important area of explainable and interpretable AI through the example of computer vision.
  arXiv  Detail & Related papers  (2022-04-19T02:57:47Z)
• A neural anisotropic view of underspecification in deep learning [60.119023683371736]
  We show that the way neural networks handle the underspecification of problems is highly dependent on the data representation. Our results highlight that understanding the architectural inductive bias in deep learning is fundamental to address the fairness, robustness, and generalization of these systems.
  arXiv  Detail & Related papers  (2021-04-29T14:31:09Z)
• Recent advances in deep learning theory [104.01582662336256]
  This paper reviews and organizes the recent advances in deep learning theory. The literature is categorized in six groups: (1) complexity and capacity-based approaches for analysing the generalizability of deep learning; (2) differential equations and their dynamic systems for modelling gradient descent and its variants; (3) the geometrical structures of the loss landscape that drives the trajectories of the dynamic systems; and (5) theoretical foundations of several special structures in network architectures.
  arXiv  Detail & Related papers  (2020-12-20T14:16:41Z)
• Developing Constrained Neural Units Over Time [81.19349325749037]
  This paper focuses on an alternative way of defining Neural Networks, that is different from the majority of existing approaches. The structure of the neural architecture is defined by means of a special class of constraints that are extended also to the interaction with data. The proposed theory is cast into the time domain, in which data are presented to the network in an ordered manner.
  arXiv  Detail & Related papers  (2020-09-01T09:07:25Z)
• Learning Connectivity of Neural Networks from a Topological Perspective [80.35103711638548]
  We propose a topological perspective to represent a network into a complete graph for analysis. By assigning learnable parameters to the edges which reflect the magnitude of connections, the learning process can be performed in a differentiable manner. This learning process is compatible with existing networks and owns adaptability to larger search spaces and different tasks.
  arXiv  Detail & Related papers  (2020-08-19T04:53:31Z)
• Understanding Deep Architectures with Reasoning Layer [60.90906477693774]
  We show that properties of the algorithm layers, such as convergence, stability, and sensitivity, are intimately related to the approximation and generalization abilities of the end-to-end model. Our theory can provide useful guidelines for designing deep architectures with reasoning layers.
  arXiv  Detail & Related papers  (2020-06-24T00:26:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.