CoinSeg: Contrast Inter- and Intra- Class Representations for Incremental Segmentation

• URL: http://arxiv.org/abs/2310.06368v1
• Date: Tue, 10 Oct 2023 07:08:49 GMT
• Title: CoinSeg: Contrast Inter- and Intra- Class Representations for Incremental Segmentation
• Authors: Zekang Zhang, Guangyu Gao, Jianbo Jiao, Chi Harold Liu, Yunchao Wei
• Abstract summary: Class incremental semantic segmentation aims to strike a balance between the model's stability and plasticity. We propose Contrast inter- and intra-class representations for Incremental (CoinSeg)
• Score: 85.13209973293229
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Class incremental semantic segmentation aims to strike a balance between the model's stability and plasticity by maintaining old knowledge while adapting to new concepts. However, most state-of-the-art methods use the freeze strategy for stability, which compromises the model's plasticity.In contrast, releasing parameter training for plasticity could lead to the best performance for all categories, but this requires discriminative feature representation.Therefore, we prioritize the model's plasticity and propose the Contrast inter- and intra-class representations for Incremental Segmentation (CoinSeg), which pursues discriminative representations for flexible parameter tuning. Inspired by the Gaussian mixture model that samples from a mixture of Gaussian distributions, CoinSeg emphasizes intra-class diversity with multiple contrastive representation centroids. Specifically, we use mask proposals to identify regions with strong objectness that are likely to be diverse instances/centroids of a category. These mask proposals are then used for contrastive representations to reinforce intra-class diversity. Meanwhile, to avoid bias from intra-class diversity, we also apply category-level pseudo-labels to enhance category-level consistency and inter-category diversity. Additionally, CoinSeg ensures the model's stability and alleviates forgetting through a specific flexible tuning strategy. We validate CoinSeg on Pascal VOC 2012 and ADE20K datasets with multiple incremental scenarios and achieve superior results compared to previous state-of-the-art methods, especially in more challenging and realistic long-term scenarios. Code is available at https://github.com/zkzhang98/CoinSeg.

Related papers

• Decoupled Data Augmentation for Improving Image Classification [37.50690945158849]
  We introduce Decoupled Data Augmentation (De-DA), which resolves the fidelity-diversity dilemma. We use generative models to modify real CDPs under controlled conditions, preserving semantic consistency. We also replace the image's CIP with inter-class variants, creating diverse CDP-CIP combinations.
  arXiv  Detail & Related papers  (2024-10-29T06:27:09Z)
• Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning [113.89327264634984]
  Few-shot class-incremental learning (FSCIL) confronts the challenge of integrating new classes into a model with minimal training samples. Traditional methods widely adopt static adaptation relying on a fixed parameter space to learn from data that arrive sequentially. We propose a dual selective SSM projector that dynamically adjusts the projection parameters based on the intermediate features for dynamic adaptation.
  arXiv  Detail & Related papers  (2024-07-08T17:09:39Z)
• Tendency-driven Mutual Exclusivity for Weakly Supervised Incremental Semantic Segmentation [56.1776710527814]
  Weakly Incremental Learning for Semantic (WILSS) leverages a pre-trained segmentation model to segment new classes using cost-effective and readily available image-level labels. A prevailing way to solve WILSS is the generation of seed areas for each new class, serving as a form of pixel-level supervision. We propose an innovative, tendency-driven relationship of mutual exclusivity, meticulously tailored to govern the behavior of the seed areas.
  arXiv  Detail & Related papers  (2024-04-18T08:23:24Z)
• Fine-grained Recognition with Learnable Semantic Data Augmentation [68.48892326854494]
  Fine-grained image recognition is a longstanding computer vision challenge. We propose diversifying the training data at the feature-level to alleviate the discriminative region loss problem. Our method significantly improves the generalization performance on several popular classification networks.
  arXiv  Detail & Related papers  (2023-09-01T11:15:50Z)
• Variational Classification [51.2541371924591]
  We derive a variational objective to train the model, analogous to the evidence lower bound (ELBO) used to train variational auto-encoders. Treating inputs to the softmax layer as samples of a latent variable, our abstracted perspective reveals a potential inconsistency. We induce a chosen latent distribution, instead of the implicit assumption found in a standard softmax layer.
  arXiv  Detail & Related papers  (2023-05-17T17:47:19Z)
• Multi-Symmetry Ensembles: Improving Diversity and Generalization via Opposing Symmetries [14.219011458423363]
  We present Multi-Symmetry Ensembles (MSE), a framework for constructing diverse ensembles by capturing the multiplicity of hypotheses along symmetry axes. MSE effectively captures the multiplicity of conflicting hypotheses that is often required in large, diverse datasets like ImageNet. As a result of their inherent diversity, MSE improves classification performance, uncertainty quantification, and generalization across a series of transfer tasks.
  arXiv  Detail & Related papers  (2023-03-04T19:11:54Z)
• Sequential Ensembling for Semantic Segmentation [4.030520171276982]
  We benchmark the popular ensembling approach of combining predictions of multiple, independently-trained, state-of-the-art models. We propose a novel method inspired by boosting to sequentially ensemble networks that significantly outperforms the naive ensemble baseline.
  arXiv  Detail & Related papers  (2022-10-08T22:13:59Z)
• SePiCo: Semantic-Guided Pixel Contrast for Domain Adaptive Semantic Segmentation [52.62441404064957]
  Domain adaptive semantic segmentation attempts to make satisfactory dense predictions on an unlabeled target domain by utilizing the model trained on a labeled source domain. Many methods tend to alleviate noisy pseudo labels, however, they ignore intrinsic connections among cross-domain pixels with similar semantic concepts. We propose Semantic-Guided Pixel Contrast (SePiCo), a novel one-stage adaptation framework that highlights the semantic concepts of individual pixel.
  arXiv  Detail & Related papers  (2022-04-19T11:16:29Z)
• A Compositional Feature Embedding and Similarity Metric for Ultra-Fine-Grained Visual Categorization [16.843126268445726]
  Fine-grained visual categorization (FGVC) aims at classifying objects with small inter-class variances. This paper proposes a novel compositional feature embedding and similarity metric ( CECS) for ultra-fine-grained visual categorization. Experimental results on two ultra-FGVC datasets and one FGVC dataset with recent benchmark methods consistently demonstrate that the proposed CECS method achieves the state-the-art performance.
  arXiv  Detail & Related papers  (2021-09-25T15:05:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.