Continual Learning for Class- and Domain-Incremental Semantic Segmentation

• URL: http://arxiv.org/abs/2209.08023v1
• Date: Fri, 16 Sep 2022 16:08:15 GMT
• Title: Continual Learning for Class- and Domain-Incremental Semantic Segmentation
• Authors: Tobias Kalb, Masoud Roschani, Miriam Ruf, J\"urgen Beyerer
• Abstract summary: The goal of our work is to evaluate and adapt established solutions for continual object recognition to the task of semantic segmentation. We show that the nature of the task of semantic segmentation changes which methods are most effective in mitigating forgetting compared to image classification.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The field of continual deep learning is an emerging field and a lot of progress has been made. However, concurrently most of the approaches are only tested on the task of image classification, which is not relevant in the field of intelligent vehicles. Only recently approaches for class-incremental semantic segmentation were proposed. However, all of those approaches are based on some form of knowledge distillation. At the moment there are no investigations on replay-based approaches that are commonly used for object recognition in a continual setting. At the same time while unsupervised domain adaption for semantic segmentation gained a lot of traction, investigations regarding domain-incremental learning in an continual setting is not well-studied. Therefore, the goal of our work is to evaluate and adapt established solutions for continual object recognition to the task of semantic segmentation and to provide baseline methods and evaluation protocols for the task of continual semantic segmentation. We firstly introduce evaluation protocols for the class- and domain-incremental segmentation and analyze selected approaches. We show that the nature of the task of semantic segmentation changes which methods are most effective in mitigating forgetting compared to image classification. Especially, in class-incremental learning knowledge distillation proves to be a vital tool, whereas in domain-incremental learning replay methods are the most effective method.

Related papers

• A Bottom-Up Approach to Class-Agnostic Image Segmentation [4.086366531569003]
  We present a novel bottom-up formulation for addressing the class-agnostic segmentation problem. We supervise our network directly on the projective sphere of its feature space. Our bottom-up formulation exhibits exceptional generalization capability, even when trained on datasets designed for class-based segmentation.
  arXiv  Detail & Related papers  (2024-09-20T17:56:02Z)
• Learning Context-aware Classifier for Semantic Segmentation [88.88198210948426]
  In this paper, contextual hints are exploited via learning a context-aware classifier. Our method is model-agnostic and can be easily applied to generic segmentation models. With only negligible additional parameters and +2% inference time, decent performance gain has been achieved on both small and large models.
  arXiv  Detail & Related papers  (2023-03-21T07:00:35Z)
• Semantics-Aware Dynamic Localization and Refinement for Referring Image Segmentation [102.25240608024063]
  Referring image segments an image from a language expression. We develop an algorithm that shifts from being localization-centric to segmentation-language. Compared to its counterparts, our method is more versatile yet effective.
  arXiv  Detail & Related papers  (2023-03-11T08:42:40Z)
• Domain-invariant Prototypes for Semantic Segmentation [30.932130453313537]
  We present an easy-to-train framework that learns domain-invariant prototypes for domain adaptive semantic segmentation. Our method involves only one-stage training and does not need to be trained on large-scale un-annotated target images.
  arXiv  Detail & Related papers  (2022-08-12T02:21:05Z)
• Pin the Memory: Learning to Generalize Semantic Segmentation [68.367763672095]
  We present a novel memory-guided domain generalization method for semantic segmentation based on meta-learning framework. Our method abstracts the conceptual knowledge of semantic classes into categorical memory which is constant beyond the domains.
  arXiv  Detail & Related papers  (2022-04-07T17:34:01Z)
• A Contrastive Distillation Approach for Incremental Semantic Segmentation in Aerial Images [15.75291664088815]
  A major issue concerning current deep neural architectures is known as catastrophic forgetting. We propose a contrastive regularization, where any given input is compared with its augmented version. We show the effectiveness of our solution on the Potsdam dataset, outperforming the incremental baseline in every test.
  arXiv  Detail & Related papers  (2021-12-07T16:44:45Z)
• Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation [19.55572909866489]
  We propose a region-based active learning approach for semantic segmentation under a domain shift. Our algorithm, Active Learning via Region Impurity and Prediction Uncertainty (AL-RIPU), introduces a novel acquisition strategy characterizing the spatial adjacency of image regions. Our method only requires very few annotations to almost reach the supervised performance and substantially outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-11-25T06:40:58Z)
• MCDAL: Maximum Classifier Discrepancy for Active Learning [74.73133545019877]
  Recent state-of-the-art active learning methods have mostly leveraged Generative Adversarial Networks (GAN) for sample acquisition. We propose in this paper a novel active learning framework that we call Maximum Discrepancy for Active Learning (MCDAL) In particular, we utilize two auxiliary classification layers that learn tighter decision boundaries by maximizing the discrepancies among them.
  arXiv  Detail & Related papers  (2021-07-23T06:57:08Z)
• Towards Adaptive Semantic Segmentation by Progressive Feature Refinement [16.40758125170239]
  We propose an innovative progressive feature refinement framework, along with domain adversarial learning to boost the transferability of segmentation networks. As a result, the segmentation models trained with source domain images can be transferred to a target domain without significant performance degradation.
  arXiv  Detail & Related papers  (2020-09-30T04:17:48Z)
• Towards Interpretable Semantic Segmentation via Gradient-weighted Class Activation Mapping [71.91734471596432]
  We propose SEG-GRAD-CAM, a gradient-based method for interpreting semantic segmentation. Our method is an extension of the widely-used Grad-CAM method, applied locally to produce heatmaps showing the relevance of individual pixels for semantic segmentation.
  arXiv  Detail & Related papers  (2020-02-26T12:32:40Z)
• Multi-Task Incremental Learning for Object Detection [71.57155077119839]
  Multi-task learns multiple tasks, while sharing knowledge and computation among them. It suffers from catastrophic forgetting of previous knowledge when learned incrementally without access to the old data.
  arXiv  Detail & Related papers  (2020-02-13T04:58:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.