Effects of structural properties of neural networks on machine learning performance

• URL: http://arxiv.org/abs/2507.10005v1
• Date: Mon, 14 Jul 2025 07:39:19 GMT
• Title: Effects of structural properties of neural networks on machine learning performance
• Authors: Yash Arya, Sang Hoon Lee,
• Abstract summary: This study contributes meaningfully to network science and machine learning, providing insights that could inspire the design of more biologically informed neural networks.<n>Our findings reveal that structural properties do affect performance to some extent. Specifically, networks featuring coherent, densely interconnected communities demonstrate enhanced learning capabilities.
• Score: 12.106994960669924
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, graph-based machine learning techniques, such as reinforcement learning and graph neural networks, have garnered significant attention. While some recent studies have started to explore the relationship between the graph structure of neural networks and their predictive performance, they often limit themselves to a narrow range of model networks, particularly lacking mesoscale structures such as communities. Our work advances this area by conducting a more comprehensive investigation, incorporating realistic network structures characterized by heterogeneous degree distributions and community structures, which are typical characteristics of many real networks. These community structures offer a nuanced perspective on network architecture. Our analysis employs model networks such as random and scale-free networks, alongside a comparison with a biological neural network and its subsets for more detailed analysis. We examine the impact of these structural attributes on the performance of image classification tasks. Our findings reveal that structural properties do affect performance to some extent. Specifically, networks featuring coherent, densely interconnected communities demonstrate enhanced learning capabilities. The comparison with the biological neural network emphasizes the relevance of our findings to real-world structures, suggesting an intriguing connection worth further exploration. This study contributes meaningfully to network science and machine learning, providing insights that could inspire the design of more biologically informed neural networks.

Related papers

• Collective variables of neural networks: empirical time evolution and scaling laws [0.535514140374842]
  We show that certain measures on the spectrum of the empirical neural tangent kernel, specifically entropy and trace, yield insight into the representations learned by a neural network. Results are demonstrated first on test cases before being shown on more complex networks, including transformers, auto-encoders, graph neural networks, and reinforcement learning studies.
  arXiv  Detail & Related papers  (2024-10-09T21:37:14Z)
• Coding schemes in neural networks learning classification tasks [52.22978725954347]
  We investigate fully-connected, wide neural networks learning classification tasks. We show that the networks acquire strong, data-dependent features. Surprisingly, the nature of the internal representations depends crucially on the neuronal nonlinearity.
  arXiv  Detail & Related papers  (2024-06-24T14:50:05Z)
• Leveraging advances in machine learning for the robust classification and interpretation of networks [0.0]
  Simulation approaches involve selecting a suitable network generative model such as Erd"os-R'enyi or small-world. We utilize advances in interpretable machine learning to classify simulated networks by our generative models based on various network attributes.
  arXiv  Detail & Related papers  (2024-03-20T00:24:23Z)
• A Network Science perspective of Graph Convolutional Networks: A survey [11.18312489268624]
  We provide a network science perspective on graph convolutional networks (GCNs) GCNs integrate node features into graph structures via neighbourhood aggregation and message passing. Our novel taxonomy classifies GCNs from three structural information angles, i.e., the layer-wise message aggregation scope, the message content, and the overall learning scope.
  arXiv  Detail & Related papers  (2023-01-12T06:03:57Z)
• The Neural Race Reduction: Dynamics of Abstraction in Gated Networks [12.130628846129973]
  We introduce the Gated Deep Linear Network framework that schematizes how pathways of information flow impact learning dynamics. We derive an exact reduction and, for certain cases, exact solutions to the dynamics of learning. Our work gives rise to general hypotheses relating neural architecture to learning and provides a mathematical approach towards understanding the design of more complex architectures.
  arXiv  Detail & Related papers  (2022-07-21T12:01:03Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Wide Neural Networks Forget Less Catastrophically [39.907197907411266]
  We study the impact of "width" of the neural network architecture on catastrophic forgetting. We study the learning dynamics of the network from various perspectives.
  arXiv  Detail & Related papers  (2021-10-21T23:49:23Z)
• Learning Contact Dynamics using Physically Structured Neural Networks [81.73947303886753]
  We use connections between deep neural networks and differential equations to design a family of deep network architectures for representing contact dynamics between objects. We show that these networks can learn discontinuous contact events in a data-efficient manner from noisy observations. Our results indicate that an idealised form of touch feedback is a key component of making this learning problem tractable.
  arXiv  Detail & Related papers  (2021-02-22T17:33:51Z)
• Learning Connectivity of Neural Networks from a Topological Perspective [80.35103711638548]
  We propose a topological perspective to represent a network into a complete graph for analysis. By assigning learnable parameters to the edges which reflect the magnitude of connections, the learning process can be performed in a differentiable manner. This learning process is compatible with existing networks and owns adaptability to larger search spaces and different tasks.
  arXiv  Detail & Related papers  (2020-08-19T04:53:31Z)
• Graph Structure of Neural Networks [104.33754950606298]
  We show how the graph structure of neural networks affect their predictive performance. A "sweet spot" of relational graphs leads to neural networks with significantly improved predictive performance. Top-performing neural networks have graph structure surprisingly similar to those of real biological neural networks.
  arXiv  Detail & Related papers  (2020-07-13T17:59:31Z)
• Neural networks adapting to datasets: learning network size and topology [77.34726150561087]
  We introduce a flexible setup allowing for a neural network to learn both its size and topology during the course of a gradient-based training. The resulting network has the structure of a graph tailored to the particular learning task and dataset.
  arXiv  Detail & Related papers  (2020-06-22T12:46:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.