Benchmarking local Hebbian learning rules for memory storage and prototype extraction

• URL: http://arxiv.org/abs/2605.01074v1
• Date: Fri, 01 May 2026 20:15:32 GMT
• Title: Benchmarking local Hebbian learning rules for memory storage and prototype extraction
• Abstract summary: Associative memory or content-addressable memory is an important component function in computer science and information processing.<n>We benchmark associative memory function of seven different Hebbian learning rules employed in non-modular and modular recurrent networks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Associative memory or content-addressable memory is an important component function in computer science and information processing, and at the same time a key concept in cognitive and computational brain science. Many different neural network architectures and learning rules have been proposed to model the brain's associative memory while investigating key component functions like figure-ground segmentation, perceptual reconstruction and rivalry. A less investigated but equally important capability of associative memory is prototype extraction where the training set comprises distorted prototype instances and the task is to recall the correct generating prototype given a new distorted instance. In this paper we benchmark associative memory function of seven different Hebbian learning rules employed in non-modular and modular recurrent networks with winner-take-all dynamics operating on moderately sparse binary patterns. We measure pattern storage and weight information capacity, prototype extraction capabilities, and sensitivity to correlations in data. The original additive Hebb rule comes out with worst capacity, covariance learning proves to be robust but with moderate capacity, and the Bayesian-Hebbian learning rules show highest capacity in almost all different conditions tested.

Related papers

• Mela: Test-Time Memory Consolidation based on Transformation Hypothesis [0.0]
  We propose a neural memory architecture composed of two functionally distinct sub-modules that operate at different update frequencies.<n>Inspired by the transformation hypothesis, the low-frequency sub- module produces high-level representations that capture abstract, gist-level knowledge.<n>We integrate HMM into a Transformer-based language decoder to form Mela, a family of memory-augmented language models that perform online memory consolidation at test time.
  arXiv  Detail & Related papers  (2026-05-11T13:20:51Z)
• The AI Hippocampus: How Far are We From Human Memory? [77.04745635827278]
  Implicit memory refers to the knowledge embedded within the internal parameters of pre-trained transformers.<n>Explicit memory involves external storage and retrieval components designed to augment model outputs with dynamic, queryable knowledge representations.<n>Agentic memory introduces persistent, temporally extended memory structures within autonomous agents.
  arXiv  Detail & Related papers  (2026-01-14T03:24:08Z)
• A Neural Network Model of Complementary Learning Systems: Pattern Separation and Completion for Continual Learning [2.9123921488295768]
  Learning new information without forgetting prior knowledge is central to human intelligence.<n>In contrast, neural network models suffer from catastrophic forgetting when acquiring new information.<n>We develop a neurally plausible continual learning model that achieves close to state-of-the-art accuracy (90%)<n>Our work provides a functional template for modeling memory consolidation, generalization, and continual learning in both biological and artificial systems.
  arXiv  Detail & Related papers  (2025-07-15T15:05:26Z)
• A Computational Model of Learning and Memory Using Structurally Dynamic Cellular Automata [0.0]
  This paper proposes a mathematical and computational model of learning and memory based on a small set of bio-plausible functions.<n> Experimental results show that the model can make near-optimal choices to re-discover a reward state after a single training run.
  arXiv  Detail & Related papers  (2024-12-20T17:26:17Z)
• Benchmarking Hebbian learning rules for associative memory [0.0]
  Associative memory is a key concept in cognitive and computational brain science.<n>We benchmark six different learning rules on storage capacity and prototype extraction.
  arXiv  Detail & Related papers  (2023-12-30T21:49:47Z)
• Brain-like combination of feedforward and recurrent network components achieves prototype extraction and robust pattern recognition [0.0]
  Associative memory has been a prominent candidate for the computation performed by the massively recurrent neocortical networks. We combine a recurrent attractor network with a feedforward network that learns distributed representations using an unsupervised Hebbian-Bayesian learning rule. We demonstrate that the recurrent attractor component implements associative memory when trained on the feedforward-driven internal (hidden) representations.
  arXiv  Detail & Related papers  (2022-06-30T06:03:11Z)
• Hierarchical Variational Memory for Few-shot Learning Across Domains [120.87679627651153]
  We introduce a hierarchical prototype model, where each level of the prototype fetches corresponding information from the hierarchical memory. The model is endowed with the ability to flexibly rely on features at different semantic levels if the domain shift circumstances so demand. We conduct thorough ablation studies to demonstrate the effectiveness of each component in our model.
  arXiv  Detail & Related papers  (2021-12-15T15:01:29Z)
• Learning to Learn Variational Semantic Memory [132.39737669936125]
  We introduce variational semantic memory into meta-learning to acquire long-term knowledge for few-shot learning. The semantic memory is grown from scratch and gradually consolidated by absorbing information from tasks it experiences. We formulate memory recall as the variational inference of a latent memory variable from addressed contents.
  arXiv  Detail & Related papers  (2020-10-20T15:05:26Z)
• Incremental Training of a Recurrent Neural Network Exploiting a Multi-Scale Dynamic Memory [79.42778415729475]
  We propose a novel incrementally trained recurrent architecture targeting explicitly multi-scale learning. We show how to extend the architecture of a simple RNN by separating its hidden state into different modules. We discuss a training algorithm where new modules are iteratively added to the model to learn progressively longer dependencies.
  arXiv  Detail & Related papers  (2020-06-29T08:35:49Z)
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain [104.38824285741248]
  Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a method where these auxiliary samples are generated on the fly, given only the model that is being trained for the assessed objective. Instead the implicit memory of learned samples within the assessed model itself is exploited.
  arXiv  Detail & Related papers  (2020-06-22T15:07:06Z)
• Encoding-based Memory Modules for Recurrent Neural Networks [79.42778415729475]
  We study the memorization subtask from the point of view of the design and training of recurrent neural networks. We propose a new model, the Linear Memory Network, which features an encoding-based memorization component built with a linear autoencoder for sequences.
  arXiv  Detail & Related papers  (2020-01-31T11:14:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.