Saturation Self-Organizing Map

• URL: http://arxiv.org/abs/2506.10680v2
• Date: Fri, 13 Jun 2025 08:14:34 GMT
• Title: Saturation Self-Organizing Map
• Authors: Igor Urbanik, Paweł Gajewski,
• Abstract summary: We introduce Saturation Self-Organizing Maps (SatSOM) to improve knowledge retention in continual learning scenarios.<n>SatSOM incorporates a novel saturation mechanism that gradually reduces the learning rate and neighborhood radius of neurons as they accumulate information.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continual learning poses a fundamental challenge for neural systems, which often suffer from catastrophic forgetting when exposed to sequential tasks. Self-Organizing Maps (SOMs), despite their interpretability and efficiency, are not immune to this issue. In this paper, we introduce Saturation Self-Organizing Maps (SatSOM)-an extension of SOMs designed to improve knowledge retention in continual learning scenarios. SatSOM incorporates a novel saturation mechanism that gradually reduces the learning rate and neighborhood radius of neurons as they accumulate information. This effectively freezes well-trained neurons and redirects learning to underutilized areas of the map.

Related papers

• Semi-parametric Memory Consolidation: Towards Brain-like Deep Continual Learning [59.35015431695172]
  We propose a novel biomimetic continual learning framework that integrates semi-parametric memory and the wake-sleep consolidation mechanism.<n>For the first time, our method enables deep neural networks to retain high performance on novel tasks while maintaining prior knowledge in real-world challenging continual learning scenarios.
  arXiv  Detail & Related papers  (2025-04-20T19:53:13Z)
• TACOS: Task Agnostic Continual Learning in Spiking Neural Networks [1.703671463296347]
  Catastrophic interference, the loss of previously learned information when learning new information, remains a major challenge in machine learning. We show that neuro-inspired mechanisms such as synaptic consolidation and metaplasticity can mitigate catastrophic interference in a spiking neural network. Our model, TACOS, combines neuromodulation with complex synaptic dynamics to enable new learning while protecting previous information.
  arXiv  Detail & Related papers  (2024-08-16T15:42:16Z)
• Simple and Effective Transfer Learning for Neuro-Symbolic Integration [50.592338727912946]
  A potential solution to this issue is Neuro-Symbolic Integration (NeSy), where neural approaches are combined with symbolic reasoning. Most of these methods exploit a neural network to map perceptions to symbols and a logical reasoner to predict the output of the downstream task. They suffer from several issues, including slow convergence, learning difficulties with complex perception tasks, and convergence to local minima. This paper proposes a simple yet effective method to ameliorate these problems.
  arXiv  Detail & Related papers  (2024-02-21T15:51:01Z)
• Neuro-mimetic Task-free Unsupervised Online Learning with Continual Self-Organizing Maps [56.827895559823126]
  Self-organizing map (SOM) is a neural model often used in clustering and dimensionality reduction. We propose a generalization of the SOM, the continual SOM, which is capable of online unsupervised learning under a low memory budget. Our results, on benchmarks including MNIST, Kuzushiji-MNIST, and Fashion-MNIST, show almost a two times increase in accuracy.
  arXiv  Detail & Related papers  (2024-02-19T19:11:22Z)
• Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learning [54.7584721943286]
  Continual Learning (CL) methods aim to enable machine learning models to learn new tasks without catastrophic forgetting of those that have been previously mastered. Existing CL approaches often keep a buffer of previously-seen samples, perform knowledge distillation, or use regularization techniques towards this goal. We propose to only activate and select sparse neurons for learning current and past tasks at any stage.
  arXiv  Detail & Related papers  (2022-02-21T13:25:03Z)
• Reducing Catastrophic Forgetting in Self Organizing Maps with Internally-Induced Generative Replay [67.50637511633212]
  A lifelong learning agent is able to continually learn from potentially infinite streams of pattern sensory data. One major historic difficulty in building agents that adapt is that neural systems struggle to retain previously-acquired knowledge when learning from new samples. This problem is known as catastrophic forgetting (interference) and remains an unsolved problem in the domain of machine learning to this day.
  arXiv  Detail & Related papers  (2021-12-09T07:11:14Z)
• Dendritic Self-Organizing Maps for Continual Learning [0.0]
  We propose a novel algorithm inspired by biological neurons, termed Dendritic-Self-Organizing Map (DendSOM) DendSOM consists of a single layer of SOMs, which extract patterns from specific regions of the input space. It outperforms classical SOMs and several state-of-the-art continual learning algorithms on benchmark datasets.
  arXiv  Detail & Related papers  (2021-10-18T14:47:19Z)
• Continual learning under domain transfer with sparse synaptic bursting [2.314558204145174]
  We introduce a system that can learn sequentially over previously unseen datasets with little forgetting over time. We find that our method learns continually under domain transfer with sparse bursts of activity in weights that are recycled across tasks.
  arXiv  Detail & Related papers  (2021-08-26T22:53:27Z)
• Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting [135.0863818867184]
  artificial neural variability (ANV) helps artificial neural networks learn some advantages from natural'' neural networks. ANV plays as an implicit regularizer of the mutual information between the training data and the learned model. It can effectively relieve overfitting, label noise memorization, and catastrophic forgetting at negligible costs.
  arXiv  Detail & Related papers  (2020-11-12T06:06:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.