Representation Compensation Networks for Continual Semantic Segmentation

• URL: http://arxiv.org/abs/2203.05402v1
• Date: Thu, 10 Mar 2022 14:48:41 GMT
• Title: Representation Compensation Networks for Continual Semantic Segmentation
• Authors: Chang-Bin Zhang, Jia-Wen Xiao, Xialei Liu, Ying-Cong Chen, Ming-Ming Cheng
• Abstract summary: We study the continual semantic segmentation problem, where the deep neural networks are required to incorporate new classes continually without catastrophic forgetting. We propose to use a structural re- parameterization mechanism, named representation compensation (RC) module, to decouple the representation learning of both old and new knowledge. We conduct experiments on two challenging continual semantic segmentation scenarios, continual class segmentation and continual domain segmentation.
• Score: 79.05769734989164
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we study the continual semantic segmentation problem, where the deep neural networks are required to incorporate new classes continually without catastrophic forgetting. We propose to use a structural re-parameterization mechanism, named representation compensation (RC) module, to decouple the representation learning of both old and new knowledge. The RC module consists of two dynamically evolved branches with one frozen and one trainable. Besides, we design a pooled cube knowledge distillation strategy on both spatial and channel dimensions to further enhance the plasticity and stability of the model. We conduct experiments on two challenging continual semantic segmentation scenarios, continual class segmentation and continual domain segmentation. Without any extra computational overhead and parameters during inference, our method outperforms state-of-the-art performance. The code is available at \url{https://github.com/zhangchbin/RCIL}.

Related papers

• Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning [113.89327264634984]
  Few-shot class-incremental learning (FSCIL) confronts the challenge of integrating new classes into a model with minimal training samples. Traditional methods widely adopt static adaptation relying on a fixed parameter space to learn from data that arrive sequentially. We propose a dual selective SSM projector that dynamically adjusts the projection parameters based on the intermediate features for dynamic adaptation.
  arXiv  Detail & Related papers  (2024-07-08T17:09:39Z)
• Memory-guided Network with Uncertainty-based Feature Augmentation for Few-shot Semantic Segmentation [12.653336728447654]
  We propose a class-shared memory (CSM) module consisting of a set of learnable memory vectors. These memory vectors learn elemental object patterns from base classes during training whilst re-encoding query features during both training and inference. We integrate CSM and UFA into representative FSS works, with experimental results on the widely-used PASCAL-5$i$ and COCO-20$i$ datasets.
  arXiv  Detail & Related papers  (2024-06-01T19:53:25Z)
• ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning [54.68180752416519]
  Panoptic segmentation is a cutting-edge computer vision task. We introduce a novel and efficient method for continual panoptic segmentation based on Visual Prompt Tuning, dubbed ECLIPSE. Our approach involves freezing the base model parameters and fine-tuning only a small set of prompt embeddings, addressing both catastrophic forgetting and plasticity.
  arXiv  Detail & Related papers  (2024-03-29T11:31:12Z)
• Understanding Imbalanced Semantic Segmentation Through Neural Collapse [81.89121711426951]
  We show that semantic segmentation naturally brings contextual correlation and imbalanced distribution among classes. We introduce a regularizer on feature centers to encourage the network to learn features closer to the appealing structure. Our method ranks 1st and sets a new record on the ScanNet200 test leaderboard.
  arXiv  Detail & Related papers  (2023-01-03T13:51:51Z)
• C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentation [20.182928938110923]
  Temporal action segmentation tags action labels for every frame in an input untrimmed video containing multiple actions in a sequence. We propose an encoder-decoder-style architecture named C2F-TCN featuring a "coarse-to-fine" ensemble of decoder outputs. We show that the architecture is flexible for both supervised and representation learning.
  arXiv  Detail & Related papers  (2022-12-20T14:53:46Z)
• Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsing [71.19528222206088]
  We propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation for face parsing. Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection. Our method achieves the new state-of-the-art performance on the Helen, CelebA-HQ, and LapaMask datasets.
  arXiv  Detail & Related papers  (2022-03-28T02:12:30Z)
• Multi-scale and Cross-scale Contrastive Learning for Semantic Segmentation [5.281694565226513]
  We apply contrastive learning to enhance the discriminative power of the multi-scale features extracted by semantic segmentation networks. By first mapping the encoder's multi-scale representations to a common feature space, we instantiate a novel form of supervised local-global constraint.
  arXiv  Detail & Related papers  (2022-03-25T01:24:24Z)
• Weakly Supervised Semantic Segmentation via Alternative Self-Dual Teaching [82.71578668091914]
  This paper establishes a compact learning framework that embeds the classification and mask-refinement components into a unified deep model. We propose a novel alternative self-dual teaching (ASDT) mechanism to encourage high-quality knowledge interaction.
  arXiv  Detail & Related papers  (2021-12-17T11:56:56Z)
• Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations [18.655840060559168]
  This paper focuses on class incremental continual learning in semantic segmentation. New categories are made available over time while previous training data is not retained. The proposed continual learning scheme shapes the latent space to reduce forgetting whilst improving the recognition of novel classes.
  arXiv  Detail & Related papers  (2021-03-10T21:02:05Z)
• Learning Sub-Patterns in Piecewise Continuous Functions [4.18804572788063]
  Most gradient descent algorithms can optimize neural networks that are sub-differentiable in their parameters. This paper focuses on the case where the discontinuities arise from distinct sub-patterns. We propose a new discontinuous deep neural network model trainable via a decoupled two-step procedure.
  arXiv  Detail & Related papers  (2020-10-29T13:44:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.