The principle of weight divergence facilitation for unsupervised pattern recognition in spiking neural networks

• URL: http://arxiv.org/abs/2104.09943v1
• Date: Tue, 20 Apr 2021 13:11:15 GMT
• Title: The principle of weight divergence facilitation for unsupervised pattern recognition in spiking neural networks
• Authors: Oleg Nikitin, Olga Lukyanova, Alex Kunin
• Abstract summary: We propose the addition to the well-known STDP synaptic plasticity rule to directs the weight modification towards the state associated with the maximal difference between the background noise and correlated signals. It is proposed, that biological synaptic straight modification is restricted by the existence and production of bio-chemical'substances' needed for plasticity development.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Parallels between the signal processing tasks and biological neurons lead to an understanding of the principles of self-organized optimization of input signal recognition. In the present paper, we discuss such similarities among biological and technical systems. We propose the addition to the well-known STDP synaptic plasticity rule to directs the weight modification towards the state associated with the maximal difference between the background noise and correlated signals. The principle of physically constrained weight growth is used as a basis for such control of the modification of the weights. It is proposed, that biological synaptic straight modification is restricted by the existence and production of bio-chemical 'substances' needed for plasticity development. In this paper, the information about the noise-to-signal ratio is used to control such a substances' production and storage and to drive the neuron's synaptic pressures towards the state with the best signal-to-noise ratio. Several experiments with different input signal regimes are considered to understand the functioning of the proposed approach.

Related papers

• PHASE-Net: Physics-Grounded Harmonic Attention System for Efficient Remote Photoplethysmography Measurement [63.007237197267834]
  Existing deep learning methods are mostly physiological monitoring and lack theoretical robustness.<n>We propose a physics-informed r paradigm derived from the Navier-Stokes equations of hemodynamics, showing that the pulse signal follows a second-order system.<n>This provides a theoretical justification for using a Temporal Conal Network (TCN)<n>Phase-Net achieves state-of-the-art performance with strong efficiency, offering a theoretically grounded and deployment-ready r solution.
  arXiv  Detail & Related papers  (2025-09-29T14:36:45Z)
• PhysioWave: A Multi-Scale Wavelet-Transformer for Physiological Signal Representation [18.978031999678507]
  A novel wavelet-based approach for physiological signal analysis is presented, aiming to capture multi-scale time-frequency features in various physiological signals.<n>Two large-scale pretrained models specific to EMG and ECG are introduced for the first time, achieving superior performance and setting new baselines in downstream tasks.<n>A unified multi-modal framework is constructed by integrating pretrained EEG model, where each modality is guided through its dedicated branch and fused via learnable weighted fusion.
  arXiv  Detail & Related papers  (2025-06-12T05:11:41Z)
• PhysLLM: Harnessing Large Language Models for Cross-Modal Remote Physiological Sensing [49.243031514520794]
  Large Language Models (LLMs) excel at capturing long-range signals due to their text-centric design.<n>PhysLLM achieves state-the-art accuracy and robustness, demonstrating superior generalization across lighting variations and motion scenarios.
  arXiv  Detail & Related papers  (2025-05-06T15:18:38Z)
• Allostatic Control of Persistent States in Spiking Neural Networks for perception and computation [79.16635054977068]
  We introduce a novel model for updating perceptual beliefs about the environment by extending the concept of Allostasis to the control of internal representations. In this paper, we focus on an application in numerical cognition, where a bump of activity in an attractor network is used as a spatial numerical representation.
  arXiv  Detail & Related papers  (2025-03-20T12:28:08Z)
• Integrate-and-Fire from a Mathematical and Signal Processing Perspective [0.40964539027092917]
  Integrate-and-Fire (IF) is an idealized model of the spike-triggering mechanism of a biological neuron. We show that IF is closely related to the concept of Send-on-Delta (SOD) as used in threshold-based sampling.
  arXiv  Detail & Related papers  (2025-01-20T12:39:12Z)
• NeoHebbian Synapses to Accelerate Online Training of Neuromorphic Hardware [0.03377254151446239]
  A novel neoHebbian artificial synapse utilizing ReRAM devices has been proposed and experimentally validated. System-level simulations, accounting for various device and system-level non-idealities, confirm that these synapses offer a robust solution for the fast, compact, and energy-efficient implementation of advanced learning rules in neuromorphic hardware.
  arXiv  Detail & Related papers  (2024-11-27T12:06:15Z)
• Heterogeneous quantization regularizes spiking neural network activity [0.0]
  We present a data-blind neuromorphic signal conditioning strategy whereby analog data are normalized and quantized into spike phase representations. We extend this mechanism by adding a data-aware calibration step whereby the range and density of the quantization weights adapt to accumulated input statistics.
  arXiv  Detail & Related papers  (2024-09-27T02:25:44Z)
• Contribute to balance, wire in accordance: Emergence of backpropagation from a simple, bio-plausible neuroplasticity rule [0.0]
  We introduce a novel neuroplasticity rule that offers a potential mechanism for implementing BP in the brain. We demonstrate mathematically that our learning rule precisely replicates BP in layered neural networks without any approximations.
  arXiv  Detail & Related papers  (2024-05-23T03:28:52Z)
• A Sampling Theory Perspective on Activations for Implicit Neural Representations [73.6637608397055]
  Implicit Neural Representations (INRs) have gained popularity for encoding signals as compact, differentiable entities. We conduct a comprehensive analysis of these activations from a sampling theory perspective. Our investigation reveals that sinc activations, previously unused in conjunction with INRs, are theoretically optimal for signal encoding.
  arXiv  Detail & Related papers  (2024-02-08T05:52:45Z)
• Correlative Information Maximization: A Biologically Plausible Approach to Supervised Deep Neural Networks without Weight Symmetry [43.584567991256925]
  We propose a new normative approach to describe the signal propagation in biological neural networks in both forward and backward directions. This framework addresses many concerns about the biological-plausibility of conventional artificial neural networks and the backpropagation algorithm. Our approach provides a natural resolution to the weight symmetry problem between forward and backward signal propagation paths.
  arXiv  Detail & Related papers  (2023-06-07T22:14:33Z)
• Spike-based local synaptic plasticity: A survey of computational models and neuromorphic circuits [1.8464222520424338]
  We review historical, bottom-up, and top-down approaches to modeling synaptic plasticity. We identify computational primitives that can support low-latency and low-power hardware implementations of spike-based learning rules.
  arXiv  Detail & Related papers  (2022-09-30T15:35:04Z)
• Understanding Weight Similarity of Neural Networks via Chain Normalization Rule and Hypothesis-Training-Testing [58.401504709365284]
  We present a weight similarity measure that can quantify the weight similarity of non-volution neural networks. We first normalize the weights of neural networks by a chain normalization rule, which is used to introduce weight-training representation learning. We extend traditional hypothesis-testing method to validate the hypothesis on the weight similarity of neural networks.
  arXiv  Detail & Related papers  (2022-08-08T19:11:03Z)
• Credit Assignment in Neural Networks through Deep Feedback Control [59.14935871979047]
  Deep Feedback Control (DFC) is a new learning method that uses a feedback controller to drive a deep neural network to match a desired output target and whose control signal can be used for credit assignment. The resulting learning rule is fully local in space and time and approximates Gauss-Newton optimization for a wide range of connectivity patterns. To further underline its biological plausibility, we relate DFC to a multi-compartment model of cortical pyramidal neurons with a local voltage-dependent synaptic plasticity rule, consistent with recent theories of dendritic processing.
  arXiv  Detail & Related papers  (2021-06-15T05:30:17Z)
• Discriminative Singular Spectrum Classifier with Applications on Bioacoustic Signal Recognition [67.4171845020675]
  We present a bioacoustic signal classifier equipped with a discriminative mechanism to extract useful features for analysis and classification efficiently. Unlike current bioacoustic recognition methods, which are task-oriented, the proposed model relies on transforming the input signals into vector subspaces. The validity of the proposed method is verified using three challenging bioacoustic datasets containing anuran, bee, and mosquito species.
  arXiv  Detail & Related papers  (2021-03-18T11:01:21Z)
• Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling [121.50704279659253]
  We propose a cross-verified feature disentangling strategy to disentangle the physiological features with non-physiological representations. We then use the distilled physiological features for robust multi-task physiological measurements. The disentangled features are finally used for the joint prediction of multiple physiological signals like average HR values and r signals.
  arXiv  Detail & Related papers  (2020-07-16T09:39:17Z)
• Optimal Learning with Excitatory and Inhibitory synapses [91.3755431537592]
  I study the problem of storing associations between analog signals in the presence of correlations. I characterize the typical learning performance in terms of the power spectrum of random input and output processes.
  arXiv  Detail & Related papers  (2020-05-25T18:25:54Z)
• Overcoming the Weight Transport Problem via Spike-Timing-Dependent Weight Inference [9.948484577581796]
  We show that the use of spike timing alone outcompetes existing biologically plausible methods for synaptic weight inference in spiking neural network models. Our proposed method is more flexible, being applicable to any spiking neuron model, is conservative in how many parameters are required for implementation and can be deployed in an online-fashion with minimal computational overhead.
  arXiv  Detail & Related papers  (2020-03-09T09:26:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.