HPCNeuroNet: A Neuromorphic Approach Merging SNN Temporal Dynamics with Transformer Attention for FPGA-based Particle Physics

• URL: http://arxiv.org/abs/2412.17571v1
• Date: Mon, 23 Dec 2024 13:44:29 GMT
• Title: HPCNeuroNet: A Neuromorphic Approach Merging SNN Temporal Dynamics with Transformer Attention for FPGA-based Particle Physics
• Authors: Murat Isik, Hiruna Vishwamith, Jonathan Naoukin, I. Can Dikmen,
• Abstract summary: HPCNeuroNet is a pioneering fusion of Spiking Neural Networks (SNNs), Transformers, and high-performance computing tailored for particle physics. At the heart of HPCNeuroNet lies the integration of the sequential dynamism inherent in SNNs with the context-aware attention capabilities of Transformers. We demonstrate that the combination of SNNs, Transformers, and FPGA-based high-performance computing in particle physics signifies a significant step forward.
• Score: 0.0
• License:
• Abstract: This paper presents the innovative HPCNeuroNet model, a pioneering fusion of Spiking Neural Networks (SNNs), Transformers, and high-performance computing tailored for particle physics, particularly in particle identification from detector responses. Our approach leverages SNNs' intrinsic temporal dynamics and Transformers' robust attention mechanisms to enhance performance when discerning intricate particle interactions. At the heart of HPCNeuroNet lies the integration of the sequential dynamism inherent in SNNs with the context-aware attention capabilities of Transformers, enabling the model to precisely decode and interpret complex detector data. HPCNeuroNet is realized through the HLS4ML framework and optimized for deployment in FPGA environments. The model accuracy and scalability are also enhanced by this architectural choice. Benchmarked against machine learning models, HPCNeuroNet showcases better performance metrics, underlining its transformative potential in high-energy physics. We demonstrate that the combination of SNNs, Transformers, and FPGA-based high-performance computing in particle physics signifies a significant step forward and provides a strong foundation for future research.

Related papers

• Combining Aggregated Attention and Transformer Architecture for Accurate and Efficient Performance of Spiking Neural Networks [44.145870290310356]
  Spiking Neural Networks have attracted significant attention in recent years due to their distinctive low-power characteristics. Transformers models, known for their powerful self-attention mechanisms and parallel processing capabilities, have demonstrated exceptional performance across various domains. Despite the significant advantages of both SNNs and Transformers, directly combining the low-power benefits of SNNs with the high performance of Transformers remains challenging.
  arXiv  Detail & Related papers  (2024-12-18T07:07:38Z)
• Deep-Unrolling Multidimensional Harmonic Retrieval Algorithms on Neuromorphic Hardware [78.17783007774295]
  This paper explores the potential of conversion-based neuromorphic algorithms for highly accurate and energy-efficient single-snapshot multidimensional harmonic retrieval. A novel method for converting the complex-valued convolutional layers and activations into spiking neural networks (SNNs) is developed. The converted SNNs achieve almost five-fold power efficiency at moderate performance loss compared to the original CNNs.
  arXiv  Detail & Related papers  (2024-12-05T09:41:33Z)
• FL-QDSNNs: Federated Learning with Quantum Dynamic Spiking Neural Networks [4.635820333232683]
  This paper introduces the Federated Learning-Quantum Dynamic Spiking Neural Networks (FL-QDSNNs) framework. Central to our framework is a novel dynamic threshold mechanism for activating quantum gates in Quantum Spiking Neural Networks (QSNNs) Our FL-QDSNNs framework has demonstrated superior accuracies-up to 94% on the Iris dataset and markedly outperforms existing Quantum Federated Learning (QFL) approaches.
  arXiv  Detail & Related papers  (2024-12-03T09:08:33Z)
• Scalable Mechanistic Neural Networks [52.28945097811129]
  We propose an enhanced neural network framework designed for scientific machine learning applications involving long temporal sequences. By reformulating the original Mechanistic Neural Network (MNN) we reduce the computational time and space complexities from cubic and quadratic with respect to the sequence length, respectively, to linear. Extensive experiments demonstrate that S-MNN matches the original MNN in precision while substantially reducing computational resources.
  arXiv  Detail & Related papers  (2024-10-08T14:27:28Z)
• A Unified Approach for Learning the Dynamics of Power System Generators and Inverter-based Resources [12.723995633698514]
  inverter-based resources (IBRs) for renewable energy integration and electrification greatly challenges power system dynamic analysis. To account for both synchronous generators (SGs) and IBRs, this work presents an approach for learning the model of an individual dynamic component.
  arXiv  Detail & Related papers  (2024-09-22T14:07:10Z)
• Parallel Proportional Fusion of Spiking Quantum Neural Network for Optimizing Image Classification [10.069224006497162]
  We introduce a novel architecture termed Parallel Proportional Fusion of Quantum and Spiking Neural Networks (PPF-QSNN) The proposed PPF-QSNN outperforms both the existing spiking neural network and the serial quantum neural network across metrics such as accuracy, loss, and robustness. This study lays the groundwork for the advancement and application of quantum advantage in artificial intelligent computations.
  arXiv  Detail & Related papers  (2024-04-01T10:35:35Z)
• SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence [51.6943465041708]
  Spiking neural networks (SNNs) aim to realize brain-inspired intelligence on neuromorphic chips with high energy efficiency. We contribute a full-stack toolkit for pre-processing neuromorphic datasets, building deep SNNs, optimizing their parameters, and deploying SNNs on neuromorphic chips.
  arXiv  Detail & Related papers  (2023-10-25T13:15:17Z)
• Astrocyte-Integrated Dynamic Function Exchange in Spiking Neural Networks [0.0]
  This paper presents an innovative methodology for improving the robustness and computational efficiency of Spiking Neural Networks (SNNs) The proposed approach integrates astrocytes, a type of glial cell prevalent in the human brain, into SNNs, creating astrocyte-augmented networks. Notably, our astrocyte-augmented SNN displays near-zero latency and theoretically infinite throughput, implying exceptional computational efficiency.
  arXiv  Detail & Related papers  (2023-09-15T08:02:29Z)
• End-to-end codesign of Hessian-aware quantized neural networks for FPGAs and ASICs [49.358119307844035]
  We develop an end-to-end workflow for the training and implementation of co-designed neural networks (NNs) This makes efficient NN implementations in hardware accessible to nonexperts, in a single open-sourced workflow. We demonstrate the workflow in a particle physics application involving trigger decisions that must operate at the 40 MHz collision rate of the Large Hadron Collider (LHC) We implement an optimized mixed-precision NN for high-momentum particle jets in simulated LHC proton-proton collisions.
  arXiv  Detail & Related papers  (2023-04-13T18:00:01Z)
• Neural Operator with Regularity Structure for Modeling Dynamics Driven by SPDEs [70.51212431290611]
  Partial differential equations (SPDEs) are significant tools for modeling dynamics in many areas including atmospheric sciences and physics. We propose the Neural Operator with Regularity Structure (NORS) which incorporates the feature vectors for modeling dynamics driven by SPDEs. We conduct experiments on various of SPDEs including the dynamic Phi41 model and the 2d Navier-Stokes equation.
  arXiv  Detail & Related papers  (2022-04-13T08:53:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.