Rethinking Hebbian Principle: Low-Dimensional Structural Projection for Unsupervised Learning

• URL: http://arxiv.org/abs/2510.14810v2
• Date: Wed, 22 Oct 2025 12:21:09 GMT
• Title: Rethinking Hebbian Principle: Low-Dimensional Structural Projection for Unsupervised Learning
• Authors: Shikuang Deng, Jiayuan Zhang, Yuhang Wu, Ting Chen, Shi Gu,
• Abstract summary: Hebbian learning is a biological principle that intuitively describes how neurons adapt their connections through repeated stimuli.<n>We introduce the Structural Projection Hebbian Representation (SPHeRe), a novel unsupervised learning method.<n> Experimental results show that SPHeRe achieves SOTA performance among unsupervised synaptic plasticity approaches.
• Score: 17.299267108673277
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hebbian learning is a biological principle that intuitively describes how neurons adapt their connections through repeated stimuli. However, when applied to machine learning, it suffers serious issues due to the unconstrained updates of the connections and the lack of accounting for feedback mediation. Such shortcomings limit its effective scaling to complex network architectures and tasks. To this end, here we introduce the Structural Projection Hebbian Representation (SPHeRe), a novel unsupervised learning method that integrates orthogonality and structural information preservation through a local auxiliary nonlinear block. The loss for structural information preservation backpropagates to the input through an auxiliary lightweight projection that conceptually serves as feedback mediation while the orthogonality constraints account for the boundedness of updating magnitude. Extensive experimental results show that SPHeRe achieves SOTA performance among unsupervised synaptic plasticity approaches on standard image classification benchmarks, including CIFAR-10, CIFAR-100, and Tiny-ImageNet. Furthermore, the method exhibits strong effectiveness in continual learning and transfer learning scenarios, and image reconstruction tasks show the robustness and generalizability of the extracted features. This work demonstrates the competitiveness and potential of Hebbian unsupervised learning rules within modern deep learning frameworks, demonstrating the possibility of efficient and biologically inspired learning algorithms without the strong dependence on strict backpropagation. Our code is available at https://github.com/brain-intelligence-lab/SPHeRe.

Related papers

• POUR: A Provably Optimal Method for Unlearning Representations via Neural Collapse [12.913395960667161]
  In computer vision, machine unlearning aims to remove the influence of specific visual concepts or training images without retraining from scratch.<n>We extend the notion of unlearning to the representation level, deriving a three-term interplay between forgetting efficacy, retention fidelity, and class separation.<n>Experiments on CIFAR-10/100 and PathMNIST demonstrate that POUR achieves effective unlearning while preserving retained knowledge, outperforming state-of-the-art unlearning methods on both classification-level and representation-level metrics.
  arXiv  Detail & Related papers  (2025-11-24T17:38:53Z)
• Catastrophic Forgetting in Kolmogorov-Arnold Networks [27.683054983159835]
  Catastrophic forgetting is a longstanding challenge in continual learning.<n>Recent architectural advances like Kolmogorov-Arnold Networks (KANs) have been suggested to offer intrinsic resistance to forgetting.<n>We present a comprehensive study of catastrophic forgetting in KANs and develop a theoretical framework that links forgetting to activation support overlap and intrinsic data dimension.
  arXiv  Detail & Related papers  (2025-11-16T23:22:50Z)
• Concept-Guided Interpretability via Neural Chunking [64.6429903327095]
  We show that neural networks exhibit patterns in their raw population activity that mirror regularities in the training data.<n>We propose three methods to extract recurring chunks on a neural population level.<n>Our work points to a new direction for interpretability, one that harnesses both cognitive principles and the structure of naturalistic data.
  arXiv  Detail & Related papers  (2025-05-16T13:49:43Z)
• Recurrent Joint Embedding Predictive Architecture with Recurrent Forward Propagation Learning [0.0]
  We introduce a vision network inspired by biological principles.<n>The network learns by predicting the representation of the next image patch (fixation) based on the sequence of past fixations.<n>We also introduce emphRecurrent-Forward propagation, a learning algorithm that avoids biologically unrealistic backpropagation through time or memory-inefficient real-time recurrent learning.
  arXiv  Detail & Related papers  (2024-11-10T01:40:42Z)
• A Unified Framework for Neural Computation and Learning Over Time [56.44910327178975]
  Hamiltonian Learning is a novel unified framework for learning with neural networks "over time" It is based on differential equations that: (i) can be integrated without the need of external software solvers; (ii) generalize the well-established notion of gradient-based learning in feed-forward and recurrent networks; (iii) open to novel perspectives.
  arXiv  Detail & Related papers  (2024-09-18T14:57:13Z)
• Unsupervised Spatial-Temporal Feature Enrichment and Fidelity Preservation Network for Skeleton based Action Recognition [20.07820929037547]
  Unsupervised skeleton based action recognition has achieved remarkable progress recently. Existing unsupervised learning methods suffer from severe overfitting problem. This paper presents an Unsupervised spatial-temporal Feature Enrichment and Fidelity Preservation framework to generate rich distributed features.
  arXiv  Detail & Related papers  (2024-01-25T09:24:07Z)
• Efficient and Flexible Neural Network Training through Layer-wise Feedback Propagation [49.44309457870649]
  Layer-wise Feedback feedback (LFP) is a novel training principle for neural network-like predictors.<n>LFP decomposes a reward to individual neurons based on their respective contributions.<n>Our method then implements a greedy reinforcing approach helpful parts of the network and weakening harmful ones.
  arXiv  Detail & Related papers  (2023-08-23T10:48:28Z)
• ConCerNet: A Contrastive Learning Based Framework for Automated Conservation Law Discovery and Trustworthy Dynamical System Prediction [82.81767856234956]
  This paper proposes a new learning framework named ConCerNet to improve the trustworthiness of the DNN based dynamics modeling. We show that our method consistently outperforms the baseline neural networks in both coordinate error and conservation metrics.
  arXiv  Detail & Related papers  (2023-02-11T21:07:30Z)
• A Robust Backpropagation-Free Framework for Images [47.97322346441165]
  We present an error kernel driven activation alignment algorithm for image data. EKDAA accomplishes through the introduction of locally derived error transmission kernels and error maps. Results are presented for an EKDAA trained CNN that employs a non-differentiable activation function.
  arXiv  Detail & Related papers  (2022-06-03T21:14:10Z)
• Minimizing Control for Credit Assignment with Strong Feedback [65.59995261310529]
  Current methods for gradient-based credit assignment in deep neural networks need infinitesimally small feedback signals. We combine strong feedback influences on neural activity with gradient-based learning and show that this naturally leads to a novel view on neural network optimization. We show that the use of strong feedback in DFC allows learning forward and feedback connections simultaneously, using a learning rule fully local in space and time.
  arXiv  Detail & Related papers  (2022-04-14T22:06:21Z)
• Learning to Modulate Random Weights: Neuromodulation-inspired Neural Networks For Efficient Continual Learning [1.9580473532948401]
  We introduce a novel neural network architecture inspired by neuromodulation in biological nervous systems. We show that this approach has strong learning performance per task despite the very small number of learnable parameters.
  arXiv  Detail & Related papers  (2022-04-08T21:12:13Z)
• Improving Music Performance Assessment with Contrastive Learning [78.8942067357231]
  This study investigates contrastive learning as a potential method to improve existing MPA systems. We introduce a weighted contrastive loss suitable for regression tasks applied to a convolutional neural network. Our results show that contrastive-based methods are able to match and exceed SoTA performance for MPA regression tasks.
  arXiv  Detail & Related papers  (2021-08-03T19:24:25Z)
• Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures [22.438735897480417]
  We study the applicability of Direct Feedback Alignment to neural view synthesis, recommender systems, geometric learning, and natural language processing. Our findings show that it successfully trains a large range of state-of-the-art deep learning architectures, with performance close to fine-tuned backpropagation.
  arXiv  Detail & Related papers  (2020-06-23T10:17:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.