One-shot learning for the long term: consolidation with an artificial hippocampal algorithm

• URL: http://arxiv.org/abs/2102.07503v1
• Date: Mon, 15 Feb 2021 12:07:26 GMT
• Title: One-shot learning for the long term: consolidation with an artificial hippocampal algorithm
• Authors: Gideon Kowadlo, Abdelrahman Ahmed, David Rawlinson
• Abstract summary: We claim that few-shot learning should be long term, assimilating knowledge for the future, without forgetting previous concepts. We tested whether an artificial hippocampal algorithm, AHA, could be used with a conventional ML model analogous to the neocortex. Results demonstrated that with the addition of AHA, the system could learn in one-shot and consolidate the knowledge for the long term without catastrophic forgetting.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Standard few-shot experiments involve learning to efficiently match previously unseen samples by class. We claim that few-shot learning should be long term, assimilating knowledge for the future, without forgetting previous concepts. In the mammalian brain, the hippocampus is understood to play a significant role in this process, by learning rapidly and consolidating knowledge to the neocortex over a short term period. In this research we tested whether an artificial hippocampal algorithm, AHA, could be used with a conventional ML model analogous to the neocortex, to achieve one-shot learning both short and long term. The results demonstrated that with the addition of AHA, the system could learn in one-shot and consolidate the knowledge for the long term without catastrophic forgetting. This study is one of the first examples of using a CLS model of hippocampus to consolidate memories, and it constitutes a step toward few-shot continual learning.

Related papers

• A New Perspective on Time Series Anomaly Detection: Faster Patch-based Broad Learning System [59.38402187365612]
  Time series anomaly detection (TSAD) has been a research hotspot in both academia and industry in recent years. Deep learning is not required for TSAD due to limitations such as slow deep learning speed. We propose Contrastive Patch-based Broad Learning System (CBLS)
  arXiv  Detail & Related papers  (2024-12-07T01:58:18Z)
• Gradual Learning: Optimizing Fine-Tuning with Partially Mastered Knowledge in Large Language Models [51.20499954955646]
  Large language models (LLMs) acquire vast amounts of knowledge from extensive text corpora during the pretraining phase. In later stages such as fine-tuning and inference, the model may encounter knowledge not covered in the initial training. We propose a two-stage fine-tuning strategy to improve the model's overall test accuracy and knowledge retention.
  arXiv  Detail & Related papers  (2024-10-08T08:35:16Z)
• HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models [24.529215038221956]
  We introduce HippoRAG, a novel retrieval framework inspired by the hippocampal indexing theory of human long-term memory. We show that our method outperforms the state-of-the-art methods remarkably, by up to 20%. Our method can tackle new types of scenarios that are out of reach of existing methods.
  arXiv  Detail & Related papers  (2024-05-23T17:47:55Z)
• Few-Shot Class-Incremental Learning with Prior Knowledge [94.95569068211195]
  We propose Learning with Prior Knowledge (LwPK) to enhance the generalization ability of the pre-trained model. Experimental results indicate that LwPK effectively enhances the model resilience against catastrophic forgetting.
  arXiv  Detail & Related papers  (2024-02-02T08:05:35Z)
• Continual Learning with Pre-Trained Models: A Survey [61.97613090666247]
  Continual Learning aims to overcome the catastrophic forgetting of former knowledge when learning new ones. This paper presents a comprehensive survey of the latest advancements in PTM-based CL.
  arXiv  Detail & Related papers  (2024-01-29T18:27:52Z)
• Continual Learning with Bayesian Model based on a Fixed Pre-trained Feature Extractor [55.9023096444383]
  Current deep learning models are characterised by catastrophic forgetting of old knowledge when learning new classes. Inspired by the process of learning new knowledge in human brains, we propose a Bayesian generative model for continual learning.
  arXiv  Detail & Related papers  (2022-04-28T08:41:51Z)
• LifeLonger: A Benchmark for Continual Disease Classification [59.13735398630546]
  We introduce LifeLonger, a benchmark for continual disease classification on the MedMNIST collection. Task and class incremental learning of diseases address the issue of classifying new samples without re-training the models from scratch. Cross-domain incremental learning addresses the issue of dealing with datasets originating from different institutions while retaining the previously obtained knowledge.
  arXiv  Detail & Related papers  (2022-04-12T12:25:05Z)
• A Novel Ontology-guided Attribute Partitioning Ensemble Learning Model for Early Prediction of Cognitive Deficits using Quantitative Structural MRI in Very Preterm Infants [3.731292216299279]
  Brain maturation and geometric features can be used with machine learning models for predicting later neurodevelopmental deficits. We developed an ensemble learning framework, which is referred to as OAP Ensemble Learning (OAP-EL) We applied the OAP-EL to predict cognitive deficits at 2 year of age using quantitative brain maturation and geometric features obtained at term equivalent age in very preterm infants.
  arXiv  Detail & Related papers  (2022-02-08T20:26:42Z)
• Mixture-of-Variational-Experts for Continual Learning [0.0]
  We propose an optimality principle that facilitates a trade-off between learning and forgetting. We propose a neural network layer for continual learning, called Mixture-of-Variational-Experts (MoVE) Our experiments on variants of the MNIST and CIFAR10 datasets demonstrate the competitive performance of MoVE layers.
  arXiv  Detail & Related papers  (2021-10-25T06:32:06Z)
• A Biologically Plausible Audio-Visual Integration Model for Continual Learning [7.680515385940673]
  We propose a novel biologically plausible audio-visual integration model (AVIM) We use multi-compartment Hodgkin-Huxley neurons to build the model and adopt the calcium-based synaptic tagging and capture as the model's learning rule. Our experimental results show that the proposed AVIM can achieve state-of-the-art continual learning performance.
  arXiv  Detail & Related papers  (2020-07-17T09:30:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.