ICICLE: Interpretable Class Incremental Continual Learning

• URL: http://arxiv.org/abs/2303.07811v2
• Date: Mon, 31 Jul 2023 14:15:03 GMT
• Title: ICICLE: Interpretable Class Incremental Continual Learning
• Authors: Dawid Rymarczyk, Joost van de Weijer, Bartosz Zieli\'nski, Bart{\l}omiej Twardowski
• Abstract summary: Interpretable Class-InCremental LEarning (ICICLE) is an exemplar-free approach that adopts a prototypical part-based approach. Our experimental results demonstrate that ICICLE reduces the interpretability concept drift and outperforms the existing exemplar-free methods of common class-incremental learning.
• Score: 35.105786309067895
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual learning enables incremental learning of new tasks without forgetting those previously learned, resulting in positive knowledge transfer that can enhance performance on both new and old tasks. However, continual learning poses new challenges for interpretability, as the rationale behind model predictions may change over time, leading to interpretability concept drift. We address this problem by proposing Interpretable Class-InCremental LEarning (ICICLE), an exemplar-free approach that adopts a prototypical part-based approach. It consists of three crucial novelties: interpretability regularization that distills previously learned concepts while preserving user-friendly positive reasoning; proximity-based prototype initialization strategy dedicated to the fine-grained setting; and task-recency bias compensation devoted to prototypical parts. Our experimental results demonstrate that ICICLE reduces the interpretability concept drift and outperforms the existing exemplar-free methods of common class-incremental learning when applied to concept-based models.

Related papers

• Temporal-Difference Variational Continual Learning [89.32940051152782]
  A crucial capability of Machine Learning models in real-world applications is the ability to continuously learn new tasks. In Continual Learning settings, models often struggle to balance learning new tasks with retaining previous knowledge. We propose new learning objectives that integrate the regularization effects of multiple previous posterior estimations.
  arXiv  Detail & Related papers  (2024-10-10T10:58:41Z)
• Exemplar-free Continual Representation Learning via Learnable Drift Compensation [24.114984920918715]
  We propose Learnable Drift Compensation (LDC), which can effectively mitigate drift in any moving backbone. LDC is fast and straightforward to integrate on top of existing continual learning approaches. We achieve state-of-the-art performance in both supervised and semi-supervised settings.
  arXiv  Detail & Related papers  (2024-07-11T14:23:08Z)
• Tendency-driven Mutual Exclusivity for Weakly Supervised Incremental Semantic Segmentation [56.1776710527814]
  Weakly Incremental Learning for Semantic (WILSS) leverages a pre-trained segmentation model to segment new classes using cost-effective and readily available image-level labels. A prevailing way to solve WILSS is the generation of seed areas for each new class, serving as a form of pixel-level supervision. We propose an innovative, tendency-driven relationship of mutual exclusivity, meticulously tailored to govern the behavior of the seed areas.
  arXiv  Detail & Related papers  (2024-04-18T08:23:24Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• Continual Learners are Incremental Model Generalizers [70.34479702177988]
  This paper extensively studies the impact of Continual Learning (CL) models as pre-trainers. We find that the transfer quality of the representation often increases gradually without noticeable degradation in fine-tuning performance. We propose a new fine-tuning scheme, GLobal Attention Discretization (GLAD), that preserves rich task-generic representation during solving downstream tasks.
  arXiv  Detail & Related papers  (2023-06-21T05:26:28Z)
• Incremental Prototype Prompt-tuning with Pre-trained Representation for Class Incremental Learning [4.717066668969749]
  Class incremental learning has attracted much attention, but most existing works still continually fine-tune the representation model. We take the pre-train-and-prompt-tuning paradigm to sequentially learn new visual concepts based on a fixed semantic rich pre-trained representation model. Our method consistently outperforms other state-of-the-art methods with a large margin.
  arXiv  Detail & Related papers  (2022-04-07T12:49:14Z)
• Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations [97.91630330328815]
  We conduct a crowdsourcing study, where participants interact with deception detection models that have been trained to distinguish between genuine and fake hotel reviews. We observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control.
  arXiv  Detail & Related papers  (2021-12-17T18:29:56Z)
• Multicriteria interpretability driven Deep Learning [0.0]
  Deep Learning methods are renowned for their performances, yet their lack of interpretability prevents them from high-stakes contexts. Recent model methods address this problem by providing post-hoc interpretability methods by reverse-engineering the model's inner workings. We propose a Multicriteria agnostic technique that allows to control the feature effects on the model's outcome by injecting knowledge in the objective function.
  arXiv  Detail & Related papers  (2021-11-28T09:41:13Z)
• Learning State Representations via Retracing in Reinforcement Learning [25.755855290244103]
  Learning via retracing is a self-supervised approach for learning the state representation for reinforcement learning tasks. We introduce Cycle-Consistency World Model (CCWM), a concrete instantiation of learning via retracing. We show that CCWM achieves state-of-the-art performance in terms of sample efficiency and performance.
  arXiv  Detail & Related papers  (2021-11-24T16:19:59Z)
• Concept Learners for Few-Shot Learning [76.08585517480807]
  We propose COMET, a meta-learning method that improves generalization ability by learning to learn along human-interpretable concept dimensions. We evaluate our model on few-shot tasks from diverse domains, including fine-grained image classification, document categorization and cell type annotation.
  arXiv  Detail & Related papers  (2020-07-14T22:04:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.