Asymmetrically connected reservoir networks learn better

• URL: http://arxiv.org/abs/2410.00584v1
• Date: Tue, 1 Oct 2024 11:15:18 GMT
• Title: Asymmetrically connected reservoir networks learn better
• Abstract summary: connectivity within the high-dimensional recurrent layer of a reservoir network is crucial for its performance. We show that connectivity within the high-dimensional recurrent layer of a reservoir network is crucial for its performance.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We show that connectivity within the high-dimensional recurrent layer of a reservoir network is crucial for its performance. To this end, we systematically investigate the impact of network connectivity on its performance, i.e., we examine the symmetry and structure of the reservoir in relation to its computational power. Reservoirs with random and asymmetric connections are found to perform better for an exemplary Mackey-Glass time series than all structured reservoirs, including biologically inspired connectivities, such as small-world topologies. This result is quantified by the information processing capacity of the different network topologies which becomes highest for asymmetric and randomly connected networks.

Related papers

• Photonic reservoir computing with complex networks [1.641989164066004]
  Photonic reservoir computing has attracted increasing attention as a fast and low-cost approach for time-series prediction.<n>The effect of the internal connection structure (network topology) on the computing performance has not been investigated for large-scale photonic reservoirs.<n>We introduce complex network structures such as small-world and scale-free network topologies of the internal nodes in the reservoir.
  arXiv  Detail & Related papers  (2026-07-25T16:39:56Z)
• ANCRe: Adaptive Neural Connection Reassignment for Efficient Depth Scaling [57.91760520589592]
  Scaling network depth has been a central driver behind the success of modern foundation models.<n>This paper revisits the default mechanism for deepening neural networks, namely residual connections.<n>We introduce adaptive neural connection reassignment (ANCRe), a principled and lightweight framework that parameterizes and learns residual connectivities from the data.
  arXiv  Detail & Related papers  (2026-02-09T18:54:18Z)
• Prediction performance of random reservoirs with different topology for nonlinear dynamical systems with different number of degrees of freedom [0.0]
  Reservoir computing (RC) is a powerful framework for predicting nonlinear dynamical systems.<n>This work investigates how the structure of the network influences the performance of RC in four systems of increasing complexity.
  arXiv  Detail & Related papers  (2025-11-27T03:24:06Z)
• Chordless cycle filtrations for dimensionality detection in complex networks via topological data analysis [2.0898985386850817]
  We introduce a novel data analysis weighting scheme for graphs, based on chordless cycles, aimed at estimating the dimensionality of networks in a data-driven way.<n>We show that the resulting descriptors can effectively estimate network dimensionality using a neural network architecture trained in a synthetic graph database.
  arXiv  Detail & Related papers  (2025-09-10T07:40:48Z)
• Dense Hebbian neural networks: a replica symmetric picture of supervised learning [4.133728123207142]
  We consider dense, associative neural-networks trained by a teacher with supervision. We investigate their computational capabilities analytically, via statistical-mechanics of spin glasses, and numerically, via Monte Carlo simulations.
  arXiv  Detail & Related papers  (2022-11-25T13:37:47Z)
• Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis [94.64007376939735]
  We theoretically characterize the impact of connectivity patterns on the convergence of deep neural networks (DNNs) under gradient descent training. We show that by a simple filtration on "unpromising" connectivity patterns, we can trim down the number of models to evaluate.
  arXiv  Detail & Related papers  (2022-05-11T17:43:54Z)
• Clustering-Based Interpretation of Deep ReLU Network [17.234442722611803]
  We recognize that the non-linear behavior of the ReLU function gives rise to a natural clustering. We propose a method to increase the level of interpretability of a fully connected feedforward ReLU neural network.
  arXiv  Detail & Related papers  (2021-10-13T09:24:11Z)
• Parallel Machine Learning for Forecasting the Dynamics of Complex Networks [0.0]
  We present a machine learning scheme for forecasting the dynamics of large complex networks. We use a parallel architecture that mimics the topology of the network of interest.
  arXiv  Detail & Related papers  (2021-08-27T06:06:41Z)
• Towards Understanding Theoretical Advantages of Complex-Reaction Networks [77.34726150561087]
  We show that a class of functions can be approximated by a complex-reaction network using the number of parameters. For empirical risk minimization, our theoretical result shows that the critical point set of complex-reaction networks is a proper subset of that of real-valued networks.
  arXiv  Detail & Related papers  (2021-08-15T10:13:49Z)
• Learning Connectivity of Neural Networks from a Topological Perspective [80.35103711638548]
  We propose a topological perspective to represent a network into a complete graph for analysis. By assigning learnable parameters to the edges which reflect the magnitude of connections, the learning process can be performed in a differentiable manner. This learning process is compatible with existing networks and owns adaptability to larger search spaces and different tasks.
  arXiv  Detail & Related papers  (2020-08-19T04:53:31Z)
• Recursive Multi-model Complementary Deep Fusion forRobust Salient Object Detection via Parallel Sub Networks [62.26677215668959]
  Fully convolutional networks have shown outstanding performance in the salient object detection (SOD) field. This paper proposes a wider'' network architecture which consists of parallel sub networks with totally different network architectures. Experiments on several famous benchmarks clearly demonstrate the superior performance, good generalization, and powerful learning ability of the proposed wider framework.
  arXiv  Detail & Related papers  (2020-08-07T10:39:11Z)
• On Power Laws in Deep Ensembles [12.739425443572202]
  We show that one large network may perform worse than an ensemble of several medium-size networks with the same total number of parameters. Using the detected power law-like dependencies, we can predict the possible gain from the ensembling of networks with given structure.
  arXiv  Detail & Related papers  (2020-07-16T17:35:32Z)
• The Heterogeneity Hypothesis: Finding Layer-Wise Differentiated Network Architectures [179.66117325866585]
  We investigate a design space that is usually overlooked, i.e. adjusting the channel configurations of predefined networks. We find that this adjustment can be achieved by shrinking widened baseline networks and leads to superior performance. Experiments are conducted on various networks and datasets for image classification, visual tracking and image restoration.
  arXiv  Detail & Related papers  (2020-06-29T17:59:26Z)
• Toward fast and accurate human pose estimation via soft-gated skip connections [97.06882200076096]
  This paper is on highly accurate and highly efficient human pose estimation. We re-analyze this design choice in the context of improving both the accuracy and the efficiency over the state-of-the-art. Our model achieves state-of-the-art results on the MPII and LSP datasets.
  arXiv  Detail & Related papers  (2020-02-25T18:51:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.