Structural features of the fly olfactory circuit mitigate the stability-plasticity dilemma in continual learning

• URL: http://arxiv.org/abs/2502.01427v1
• Date: Mon, 03 Feb 2025 15:06:11 GMT
• Title: Structural features of the fly olfactory circuit mitigate the stability-plasticity dilemma in continual learning
• Authors: Heming Zou, Yunliang Zang, Xiangyang Ji,
• Abstract summary: We introduce the fly olfactory circuit as a plug-and-play component, termed the Fly Model, which can integrate with modern machine learning methods. Our findings demonstrate that the Fly Model enhances both memory stability and learning plasticity, overcoming the limitations of current continual learning strategies.
• Score: 46.74846593421828
• License:
• Abstract: Artificial neural networks face the stability-plasticity dilemma in continual learning, while the brain can maintain memories and remain adaptable. However, the biological strategies for continual learning and their potential to inspire learning algorithms in neural networks are poorly understood. This study presents a minimal model of the fly olfactory circuit to investigate the biological strategies that support continual odor learning. We introduce the fly olfactory circuit as a plug-and-play component, termed the Fly Model, which can integrate with modern machine learning methods to address this dilemma. Our findings demonstrate that the Fly Model enhances both memory stability and learning plasticity, overcoming the limitations of current continual learning strategies. We validated its effectiveness across various challenging continual learning scenarios using commonly used datasets. The fly olfactory system serves as an elegant biological circuit for lifelong learning, offering a module that enhances continual learning with minimal additional computational cost for machine learning.

Related papers

• Contrastive Learning in Memristor-based Neuromorphic Systems [55.11642177631929]
  Spiking neural networks have become an important family of neuron-based models that sidestep many of the key limitations facing modern-day backpropagation-trained deep networks. In this work, we design and investigate a proof-of-concept instantiation of contrastive-signal-dependent plasticity (CSDP), a neuromorphic form of forward-forward-based, backpropagation-free learning.
  arXiv  Detail & Related papers  (2024-09-17T04:48:45Z)
• Emulating Brain-like Rapid Learning in Neuromorphic Edge Computing [3.735012564657653]
  Digital neuromorphic technology simulates the neural and synaptic processes of the brain using two stages of learning. We demonstrate our approach using event-driven vision sensor data and the Intel Loihi neuromorphic processor with its plasticity dynamics. Our methodology can be deployed with arbitrary plasticity models and can be applied to situations demanding quick learning and adaptation at the edge.
  arXiv  Detail & Related papers  (2024-08-28T13:51:52Z)
• Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence [59.11038175596807]
  Continual learning aims to empower artificial intelligence with strong adaptability to the real world. Existing advances mainly focus on preserving memory stability to overcome catastrophic forgetting. We propose a generic approach that appropriately attenuates old memories in parameter distributions to improve learning plasticity.
  arXiv  Detail & Related papers  (2023-08-29T02:43:58Z)
• Improving Performance in Continual Learning Tasks using Bio-Inspired Architectures [4.2903672492917755]
  We develop a biologically inspired lightweight neural network architecture that incorporates synaptic plasticity mechanisms and neuromodulation. Our approach leads to superior online continual learning performance on Split-MNIST, Split-CIFAR-10, and Split-CIFAR-100 datasets. We further demonstrate the effectiveness of our approach by integrating key design concepts into other backpropagation-based continual learning algorithms.
  arXiv  Detail & Related papers  (2023-08-08T19:12:52Z)
• Meta-Learning in Spiking Neural Networks with Reward-Modulated STDP [2.179313476241343]
  We propose a bio-plausible meta-learning model inspired by the hippocampus and the prefrontal cortex. Our new model can easily be applied to spike-based neuromorphic devices and enables fast learning in neuromorphic hardware.
  arXiv  Detail & Related papers  (2023-06-07T13:08:46Z)
• Online Distillation with Continual Learning for Cyclic Domain Shifts [52.707212371912476]
  We propose a solution by leveraging the power of continual learning methods to reduce the impact of domain shifts. Our work represents an important step forward in the field of online distillation and continual learning, with the potential to significantly impact real-world applications.
  arXiv  Detail & Related papers  (2023-04-03T11:15:05Z)
• Adaptively Integrated Knowledge Distillation and Prediction Uncertainty for Continual Learning [71.43841235954453]
  Current deep learning models often suffer from catastrophic forgetting of old knowledge when continually learning new knowledge. Existing strategies to alleviate this issue often fix the trade-off between keeping old knowledge (stability) and learning new knowledge (plasticity)
  arXiv  Detail & Related papers  (2023-01-18T05:36:06Z)
• The least-control principle for learning at equilibrium [65.2998274413952]
  We present a new principle for learning equilibrium recurrent neural networks, deep equilibrium models, or meta-learning. Our results shed light on how the brain might learn and offer new ways of approaching a broad class of machine learning problems.
  arXiv  Detail & Related papers  (2022-07-04T11:27:08Z)
• A Biologically Plausible Audio-Visual Integration Model for Continual Learning [7.680515385940673]
  We propose a novel biologically plausible audio-visual integration model (AVIM) We use multi-compartment Hodgkin-Huxley neurons to build the model and adopt the calcium-based synaptic tagging and capture as the model's learning rule. Our experimental results show that the proposed AVIM can achieve state-of-the-art continual learning performance.
  arXiv  Detail & Related papers  (2020-07-17T09:30:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.