Low-Latency Activation-Regularized Sparse Neural Operators with Distillation Assistance Towards Real-Time Edge-Deployable Virtual Sensing

• URL: http://arxiv.org/abs/2608.23987v1
• Date: Tue, 25 Aug 2026 02:26:32 GMT
• Title: Low-Latency Activation-Regularized Sparse Neural Operators with Distillation Assistance Towards Real-Time Edge-Deployable Virtual Sensing
• Abstract summary: Spiking neurons and neuromorphic computing can improve efficiency, yet surrogate-gradient training and multi-step spiking introduce convergence and latency challenges.<n>We propose the Sparse-Activation-ReLU (SAR) layer, a single-step alternative that promotes activation sparsity without surrogate-gradient training.<n>These approaches reduce L2 error by more than twofold and nearly sevenfold, respectively, while reducing spiking and spatial aggregation.
• Score: 1.2744523252873352
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Virtual sensing enables digital twins and safety-critical systems to reconstruct and forecast spatial-temporal physics in real time. However, conventional computational and data-driven methods often face challenges in generalization, latency, and energy efficiency for edge deployment. Neural operators offer a promising alternative but remain reliant on power-intensive hardware. Spiking neurons and neuromorphic computing can improve efficiency, yet surrogate-gradient training and multi-step spiking introduce convergence and latency challenges. We propose the Sparse-Activation-ReLU (SAR) layer, a single-step alternative that promotes activation sparsity without surrogate-gradient training while remaining compatible with event-based computing. Within a trunk-based NOMAD architecture, SAR achieves over a fivefold improvement in the combined Latency-Error-Energy (LEE) metric compared with Variable Spiking Neuron (VSN) and Leaky Integrate-and-Fire (LIF) implementations. We further analyze spiking entropy and feature usage and introduce synthetic knowledge distillation, reducing the LEE score by more than twofold. Finally, we improve VSN through a ReLU-based spiking loss and graph-neighbor thresholding. On the Heat Exchanger dataset, these approaches reduce L2 error by more than twofold and nearly sevenfold, respectively, while reducing spiking and spatial aggregation. Overall, the work presented is a step towards energy-efficient virtual sensing by providing an alternative framework that can be positioned towards neuromorphic or other edge device integration that can be a gold standard to compare latency, energy, and error performance for future efficient designs that are sparsity or brain-inspired spiking based.

Related papers

• Reducing ANN-SNN Conversion Error via Residual Membrane Potential Alignment [52.910770415431244]
  Spiking Neural Networks (SNNs) serve as core architectures for neuromorphic computing.<n>ANN-SNN conversion circumvents issues by reusing well-trained ANN weights for low-latency, energy-efficient inference.<n>Existing conversion schemes suffer from severe accuracy drops at small timesteps, large inference delays and cumulative quantization errors.
  arXiv  Detail & Related papers  (2026-08-14T04:46:03Z)
• WiFuse: An Attention Mechanism for Human Activity Recognition using Fused CSI Amplitude and Delay-Doppler Channel Features [1.0554048699217669]
  WiFuse is a dual-stream Channel State Information framework for human activity recognition.<n>It pairs amplitude and Delay-Doppler representations within a dual-stream strategy, enhanced by transfer learning.<n>It reaches an overall accuracy of up to 95.28% across the four environments of the XRF55 dataset and up to 98.20% on the multi-user Wi-MIR dataset.
  arXiv  Detail & Related papers  (2026-08-01T12:54:09Z)
• ReSCom: A Reconfigurable Spiking Neural Network Accelerator Using Stochastic Computing [0.0]
  ReSCom is a reconfigurable SNN accelerator that reduces hardware complexity while maintaining stable inference.<n>A unified reconfigurable design supports Integrate-and-Fire, Leaky-and-Fire, and Synaptic neuron models within a single hardware framework.
  arXiv  Detail & Related papers  (2026-06-11T16:44:08Z)
• Where to Refine, When to Stop: Rethinking Redundancy via Latent Discrepancy for Efficient Visual Autoregressive Generation [50.139984798361375]
  We propose a training-free framework that removes redundancy via latent discrepancy.<n> LD-Pruning substantially reduces inference while maintaining high generation quality, achieving up to 2.35x speedup on Infinity-8B.
  arXiv  Detail & Related papers  (2026-05-29T19:34:39Z)
• Neuroscience Inspired Graph Operators Towards Edge-Deployable Virtual Sensing for Irregular Geometries [4.190153363488523]
  Spiking neuron integration can provide a potential solution when integrated on neuromorphic hardware.<n>Current existing neuron models result in severe performance degradation towards regression-based virtual sensing.<n>We present the Variable Spiking Graph Neural Operator (VS-GNO) which integrates a sophisticated spectral-error convolutional analysis and energy-error balance loss function.
  arXiv  Detail & Related papers  (2026-04-17T21:53:20Z)
• Towards Green Wearable Computing: A Physics-Aware Spiking Neural Network for Energy-Efficient IMU-based Human Activity Recognition [0.22618128882498598]
  Spiking Neural Networks (SNNs) offer extreme event-driven energy efficiency, but standard architectures struggle with complex biomechanical topologies and temporal gradient degradation.<n>We propose Physics-Aware Spiking Neural Network (PAS-Net), a fully multiplier-free architecture explicitly tailored for Green HAR.<n>PAS-Net achieves state-of-the-art accuracy while replacing dense operations with sparse 0.1 pJ integer accumulations.
  arXiv  Detail & Related papers  (2026-04-12T04:52:52Z)
• Ge$^\text{2}$mS-T: Multi-Dimensional Grouping for Ultra-High Energy Efficiency in Spiking Transformer [84.8831358775386]
  Spiking Neural Networks (SNNs) offer superior energy efficiency over Artificial Neural Networks (ANNs)<n>Existing paradigms including ANN-SNN Conversion and Spatial-Temporal Backpropagation (STBP) suffer from inherent limitations.<n>We propose Ge$text2$mS-T, a novel architecture implementing grouped computation across temporal, spatial and network structure dimensions.
  arXiv  Detail & Related papers  (2026-04-10T02:58:46Z)
• A Latency Coding Framework for Deep Spiking Neural Networks with Ultra-Low Latency [51.75891259753546]
  Spiking neural networks (SNNs) offer a biologically inspired computing paradigm with significant potential for energy-efficient neural processing.<n>Time-To-First-Spike (TTFS) coding encodes information through the precise timing of a neuron's first spike.<n>TTFS models lack efficient training methods, suffering from high inference latency and limited performance.<n>We present a comprehensive framework, which enables the efficient training of deep TTFS-coded SNNs by employing backpropagation throuh time (BPTT) algorithm.
  arXiv  Detail & Related papers  (2026-03-24T13:52:03Z)
• USEFUSE: Uniform Stride for Enhanced Performance in Fused Layer Architecture of Deep Neural Networks [0.6435156676256051]
  This study presents the Sum-of-Products (SOP) units for convolution, which utilize low-latency left-to-right bit-serial arithmetic.<n>An effective mechanism detects and skips inefficient convolutions after ReLU layers, minimizing power consumption.<n>Two designs cater to varied demands: one focuses on minimal response time for mission-critical applications, and another focuses on resource-constrained devices with comparable latency.
  arXiv  Detail & Related papers  (2024-12-18T11:04:58Z)
• Deep-Unrolling Multidimensional Harmonic Retrieval Algorithms on Neuromorphic Hardware [78.17783007774295]
  This paper explores the potential of conversion-based neuromorphic algorithms for highly accurate and energy-efficient single-snapshot multidimensional harmonic retrieval.<n>A novel method for converting the complex-valued convolutional layers and activations into spiking neural networks (SNNs) is developed.<n>The converted SNNs achieve almost five-fold power efficiency at moderate performance loss compared to the original CNNs.
  arXiv  Detail & Related papers  (2024-12-05T09:41:33Z)
• Low-power event-based face detection with asynchronous neuromorphic hardware [2.0774873363739985]
  We present the first instance of an on-chip spiking neural network for event-based face detection deployed on the SynSense Speck neuromorphic chip. We show how to reduce precision discrepancies between off-chip clock-driven simulation used for training and on-chip event-driven inference. We achieve an on-chip face detection mAP[0.5] of 0.6 while consuming only 20 mW.
  arXiv  Detail & Related papers  (2023-12-21T19:23:02Z)
• Efficient and Flexible Neural Network Training through Layer-wise Feedback Propagation [49.44309457870649]
  Layer-wise Feedback feedback (LFP) is a novel training principle for neural network-like predictors.<n>LFP decomposes a reward to individual neurons based on their respective contributions.<n>Our method then implements a greedy reinforcing approach helpful parts of the network and weakening harmful ones.
  arXiv  Detail & Related papers  (2023-08-23T10:48:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.