Scalable Construction of Spiking Neural Networks using up to thousands of GPUs

• URL: http://arxiv.org/abs/2512.09502v1
• Date: Wed, 10 Dec 2025 10:27:31 GMT
• Title: Scalable Construction of Spiking Neural Networks using up to thousands of GPUs
• Authors: Bruno Golosio, Gianmarco Tiddia, José Villamar, Luca Pontisso, Luca Sergi, Francesco Simula, Pooja Babu, Elena Pastorelli, Abigail Morrison, Markus Diesmann, Alessandro Lonardo, Pier Stanislao Paolucci, Johanna Senk,
• Abstract summary: Simulating complex systems at scale on high-performance computing clusters demands efficient management of communication and memory.<n>We study the simulation of large-scale spiking neural networks for computational neuroscience research.
• Score: 29.200909457477007
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diverse scientific and engineering research areas deal with discrete, time-stamped changes in large systems of interacting delay differential equations. Simulating such complex systems at scale on high-performance computing clusters demands efficient management of communication and memory. Inspired by the human cerebral cortex -- a sparsely connected network of $\mathcal{O}(10^{10})$ neurons, each forming $\mathcal{O}(10^{3})$--$\mathcal{O}(10^{4})$ synapses and communicating via short electrical pulses called spikes -- we study the simulation of large-scale spiking neural networks for computational neuroscience research. This work presents a novel network construction method for multi-GPU clusters and upcoming exascale supercomputers using the Message Passing Interface (MPI), where each process builds its local connectivity and prepares the data structures for efficient spike exchange across the cluster during state propagation. We demonstrate scaling performance of two cortical models using point-to-point and collective communication, respectively.

Related papers

• Multi Part Deployment of Neural Network [0.17205106391379024]
  This paper proposes a distributed system architecture that partitions a neural network across multiple servers.<n>A Multi-Part Neural Network Execution Engine facilitates seamless execution and training across distributed partitions.<n>A Neuron Distributor module enables flexible partitioning strategies based on neuron count, percentage, identifiers, or network layers.
  arXiv  Detail & Related papers  (2025-06-02T07:24:29Z)
• Event-based backpropagation on the neuromorphic platform SpiNNaker2 [1.0597501054401728]
  EventProp is an algorithm for event-based backpropagation in spiking neural networks (SNNs)<n>Our implementation computes multi-layer networks of leaky integrate-and-fire neurons using discretized versions of the differential equations and their adjoints.<n>We demonstrate a proof-of-concept of batch-parallelized, on-chip training of SNNs using the Yin Yang dataset.
  arXiv  Detail & Related papers  (2024-12-19T16:31:42Z)
• Neuromorphic Wireless Split Computing with Multi-Level Spikes [69.73249913506042]
  Neuromorphic computing uses spiking neural networks (SNNs) to perform inference tasks.<n> embedding a small payload within each spike exchanged between spiking neurons can enhance inference accuracy without increasing energy consumption.<n> split computing - where an SNN is partitioned across two devices - is a promising solution.<n>This paper presents the first comprehensive study of a neuromorphic wireless split computing architecture that employs multi-level SNNs.
  arXiv  Detail & Related papers  (2024-11-07T14:08:35Z)
• Runtime Construction of Large-Scale Spiking Neuronal Network Models on GPU Devices [0.0]
  We propose a new method for creating network connections interactively, dynamically, and directly in GPU memory. We validate the simulation performance with both consumer and data center GPUs on two neuroscientifically relevant models. Both network construction and simulation times are comparable or shorter than those obtained with other state-of-the-art simulation technologies.
  arXiv  Detail & Related papers  (2023-06-16T14:08:27Z)
• Deep Cellular Recurrent Network for Efficient Analysis of Time-Series Data with Spatial Information [52.635997570873194]
  This work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to process complex multi-dimensional time series data with spatial information. The proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.
  arXiv  Detail & Related papers  (2021-01-12T20:08:18Z)
• Solving Mixed Integer Programs Using Neural Networks [57.683491412480635]
  This paper applies learning to the two key sub-tasks of a MIP solver, generating a high-quality joint variable assignment, and bounding the gap in objective value between that assignment and an optimal one. Our approach constructs two corresponding neural network-based components, Neural Diving and Neural Branching, to use in a base MIP solver such as SCIP. We evaluate our approach on six diverse real-world datasets, including two Google production datasets and MIPLIB, by training separate neural networks on each.
  arXiv  Detail & Related papers  (2020-12-23T09:33:11Z)
• Reservoir Memory Machines as Neural Computers [70.5993855765376]
  Differentiable neural computers extend artificial neural networks with an explicit memory without interference. We achieve some of the computational capabilities of differentiable neural computers with a model that can be trained very efficiently.
  arXiv  Detail & Related papers  (2020-09-14T12:01:30Z)
• Fast simulations of highly-connected spiking cortical models using GPUs [0.0]
  We present a library for large-scale simulations of spiking neural network models written in the C++ programming languages. We will show that the proposed library achieves state-of-the-art performance in terms of simulation time per second of biological activity.
  arXiv  Detail & Related papers  (2020-07-28T13:58:50Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)
• Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks [50.42141893913188]
  We study a distributed variable for large-scale AUC for a neural network as with a deep neural network. Our model requires a much less number of communication rounds and still a number of communication rounds in theory. Our experiments on several datasets show the effectiveness of our theory and also confirm our theory.
  arXiv  Detail & Related papers  (2020-05-05T18:08:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.