FuTCR: Future-Targeted Contrast and Repulsion for Continual Panoptic Segmentation

• URL: http://arxiv.org/abs/2605.12451v2
• Date: Mon, 18 May 2026 15:31:13 GMT
• Title: FuTCR: Future-Targeted Contrast and Repulsion for Continual Panoptic Segmentation
• Abstract summary: We propose a framework that restructures representations before new classes are introduced.<n>FuTCR improves relative new-class panoptic quality over the state-of-the-art by up to 28%.
• Score: 25.83154341103521
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Continual Panoptic Segmentation (CPS) requires methods that can quickly adapt to new categories over time. The nature of this dense prediction task means that training images may contain a mix of labeled and unlabeled objects. As nothing is known about these unlabeled objects a priori, existing methods often simply group any unlabeled pixel into a single "background" class during training. In effect, during training, they repeatedly tell the model that all the different background categories are the same (even when they aren't). This makes learning to identify different background categories as they are added challenging since these new categories may require using information the model was previously told was unimportant and ignored. Thus, we propose a Future-Targeted Contrastive and Repulsive (FuTCR) framework that addresses this limitation by restructuring representations before new classes are introduced. FuTCR first discovers confident future-like regions by grouping model-predicted masks whose pixels are consistently classified as background but exhibit non-background logits. Next, FuTCR applies pixel-to-region contrast to build coherent prototypes from these unlabeled regions, while simultaneously repelling background features away from known-class prototypes to explicitly reserve representational space for future categories. Experiments across six CPS settings and a range of dataset sizes show FuTCR improves relative new-class panoptic quality over the state-of-the-art by up to 28%, while preserving or improving base-class performance with gains up to 4%.

Related papers

• BACS: Background Aware Continual Semantic Segmentation [15.821935479975343]
  In autonomous driving, there's a need to incorporate new classes as the operating environment of the deployed agent becomes more complex. For enhanced annotation efficiency, ideally, only pixels belonging to new classes would be annotated. This paper proposes a Backward Background Shift Detector (BACS) to detect previously observed classes.
  arXiv  Detail & Related papers  (2024-04-19T19:25:26Z)
• DiffusePast: Diffusion-based Generative Replay for Class Incremental Semantic Segmentation [73.54038780856554]
  Class Incremental Semantic (CISS) extends the traditional segmentation task by incrementally learning newly added classes. Previous work has introduced generative replay, which involves replaying old class samples generated from a pre-trained GAN. We propose DiffusePast, a novel framework featuring a diffusion-based generative replay module that generates semantically accurate images with more reliable masks guided by different instructions.
  arXiv  Detail & Related papers  (2023-08-02T13:13:18Z)
• Evidential Deep Learning for Class-Incremental Semantic Segmentation [15.563703446465823]
  Class-Incremental Learning is a challenging problem in machine learning that aims to extend previously trained neural networks with new classes. In this paper, we address the problem of how to model unlabeled classes while avoiding spurious feature clustering of future uncorrelated classes. Our method factorizes the problem into a separate foreground class probability, calculated by the expected value of the Dirichlet distribution, and an unknown class (background) probability corresponding to the uncertainty of the estimate.
  arXiv  Detail & Related papers  (2022-12-06T10:13:30Z)
• Few-shot Open-set Recognition Using Background as Unknowns [58.04165813493666]
  Few-shot open-set recognition aims to classify both seen and novel images given only limited training data of seen classes. Our proposed method not only outperforms multiple baselines but also sets new results on three popular benchmarks.
  arXiv  Detail & Related papers  (2022-07-19T04:19:29Z)
• A Simple Baseline for Zero-shot Semantic Segmentation with Pre-trained Vision-language Model [61.58071099082296]
  It is unclear how to make zero-shot recognition working well on broader vision problems, such as object detection and semantic segmentation. In this paper, we target for zero-shot semantic segmentation, by building it on an off-the-shelf pre-trained vision-language model, i.e., CLIP. Our experimental results show that this simple framework surpasses previous state-of-the-arts by a large margin.
  arXiv  Detail & Related papers  (2021-12-29T18:56:18Z)
• Rectifying the Shortcut Learning of Background: Shared Object Concentration for Few-Shot Image Recognition [101.59989523028264]
  Few-Shot image classification aims to utilize pretrained knowledge learned from a large-scale dataset to tackle a series of downstream classification tasks. We propose COSOC, a novel Few-Shot Learning framework, to automatically figure out foreground objects at both pretraining and evaluation stage.
  arXiv  Detail & Related papers  (2021-07-16T07:46:41Z)
• A Closer Look at Self-training for Zero-Label Semantic Segmentation [53.4488444382874]
  Being able to segment unseen classes not observed during training is an important technical challenge in deep learning. Prior zero-label semantic segmentation works approach this task by learning visual-semantic embeddings or generative models. We propose a consistency regularizer to filter out noisy pseudo-labels by taking the intersections of the pseudo-labels generated from different augmentations of the same image.
  arXiv  Detail & Related papers  (2021-04-21T14:34:33Z)
• Background Splitting: Finding Rare Classes in a Sea of Background [55.03789745276442]
  We focus on the real-world problem of training accurate deep models for image classification of a small number of rare categories. In these scenarios, almost all images belong to the background category in the dataset (>95% of the dataset is background) We demonstrate that both standard fine-tuning approaches and state-of-the-art approaches for training on imbalanced datasets do not produce accurate deep models in the presence of this extreme imbalance.
  arXiv  Detail & Related papers  (2020-08-28T23:05:15Z)
• Modeling the Background for Incremental Learning in Semantic Segmentation [39.025848280224785]
  Deep architectures are vulnerable to catastrophic forgetting. This paper addresses this problem in the context of semantic segmentation. We propose a new distillation-based framework which explicitly accounts for this shift.
  arXiv  Detail & Related papers  (2020-02-03T13:30:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.