CLEO: Continual Learning of Evolving Ontologies

• URL: http://arxiv.org/abs/2407.08411v1
• Date: Thu, 11 Jul 2024 11:32:33 GMT
• Title: CLEO: Continual Learning of Evolving Ontologies
• Authors: Shishir Muralidhara, Saqib Bukhari, Georg Schneider, Didier Stricker, René Schuster,
• Abstract summary: Continual learning (CL) aims to instill the lifelong learning of humans in intelligent systems. General learning processes are not just limited to learning information, but also refinement of existing information. CLEO is motivated by the need for intelligent systems to adapt to real-world changes over time.
• Score: 12.18795037817058
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continual learning (CL) addresses the problem of catastrophic forgetting in neural networks, which occurs when a trained model tends to overwrite previously learned information, when presented with a new task. CL aims to instill the lifelong learning characteristic of humans in intelligent systems, making them capable of learning continuously while retaining what was already learned. Current CL problems involve either learning new domains (domain-incremental) or new and previously unseen classes (class-incremental). However, general learning processes are not just limited to learning information, but also refinement of existing information. In this paper, we define CLEO - Continual Learning of Evolving Ontologies, as a new incremental learning setting under CL to tackle evolving classes. CLEO is motivated by the need for intelligent systems to adapt to real-world ontologies that change over time, such as those in autonomous driving. We use Cityscapes, PASCAL VOC, and Mapillary Vistas to define the task settings and demonstrate the applicability of CLEO. We highlight the shortcomings of existing CIL methods in adapting to CLEO and propose a baseline solution, called Modelling Ontologies (MoOn). CLEO is a promising new approach to CL that addresses the challenge of evolving ontologies in real-world applications. MoOn surpasses previous CL approaches in the context of CLEO.

Related papers

• Continual Task Learning through Adaptive Policy Self-Composition [54.95680427960524]
  CompoFormer is a structure-based continual transformer model that adaptively composes previous policies via a meta-policy network. Our experiments reveal that CompoFormer outperforms conventional continual learning (CL) methods, particularly in longer task sequences.
  arXiv  Detail & Related papers  (2024-11-18T08:20:21Z)
• CLoG: Benchmarking Continual Learning of Image Generation Models [29.337710309698515]
  This paper advocates for shifting the research focus from classification-based CL to CLoG. We adapt three types of existing CL methodologies, replay-based, regularization-based, and parameter-isolation-based methods to generative tasks. Our benchmarks and results yield intriguing insights that can be valuable for developing future CLoG methods.
  arXiv  Detail & Related papers  (2024-06-07T02:12:29Z)
• Recent Advances of Foundation Language Models-based Continual Learning: A Survey [31.171203978742447]
  Foundation language models (LMs) have marked significant achievements in the domains of natural language processing (NLP) and computer vision (CV) However, they can not emulate human-like continuous learning due to catastrophic forgetting. Various continual learning (CL)-based methodologies have been developed to refine LMs, enabling them to adapt to new tasks without forgetting previous knowledge.
  arXiv  Detail & Related papers  (2024-05-28T23:32:46Z)
• A Unified and General Framework for Continual Learning [58.72671755989431]
  Continual Learning (CL) focuses on learning from dynamic and changing data distributions while retaining previously acquired knowledge. Various methods have been developed to address the challenge of catastrophic forgetting, including regularization-based, Bayesian-based, and memory-replay-based techniques. This research aims to bridge this gap by introducing a comprehensive and overarching framework that encompasses and reconciles these existing methodologies.
  arXiv  Detail & Related papers  (2024-03-20T02:21:44Z)
• POP: Prompt Of Prompts for Continual Learning [59.15888651733645]
  Continual learning (CL) aims to mimic the human ability to learn new concepts without catastrophic forgetting. We show that a foundation model equipped with POP learning is able to outperform classic CL methods by a significant margin.
  arXiv  Detail & Related papers  (2023-06-14T02:09:26Z)
• Beyond Supervised Continual Learning: a Review [69.9674326582747]
  Continual Learning (CL) is a flavor of machine learning where the usual assumption of stationary data distribution is relaxed or omitted. Changes in the data distribution can cause the so-called catastrophic forgetting (CF) effect: an abrupt loss of previous knowledge. This article reviews literature that study CL in other settings, such as learning with reduced supervision, fully unsupervised learning, and reinforcement learning.
  arXiv  Detail & Related papers  (2022-08-30T14:44:41Z)
• Continual Lifelong Learning in Natural Language Processing: A Survey [3.9103337761169943]
  Continual learning (CL) aims to enable information systems to learn from a continuous data stream across time. It is difficult for existing deep learning architectures to learn a new task without largely forgetting previously acquired knowledge. We look at the problem of CL through the lens of various NLP tasks.
  arXiv  Detail & Related papers  (2020-12-17T18:44:36Z)
• A Survey on Curriculum Learning [48.36129047271622]
  Curriculum learning (CL) is a training strategy that trains a machine learning model from easier data to harder data. As an easy-to-use plug-in, the CL strategy has demonstrated its power in improving the generalization capacity and convergence rate of various models.
  arXiv  Detail & Related papers  (2020-10-25T17:15:04Z)
• Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning [74.07455280246212]
  Continual learning studies agents that learn from streams of tasks without forgetting previous ones while adapting to new ones. We show that current continual learning, meta-learning, meta-continual learning, and continual-meta learning techniques fail in this new scenario. We propose Continual-MAML, an online extension of the popular MAML algorithm as a strong baseline for this scenario.
  arXiv  Detail & Related papers  (2020-03-12T15:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.