CFSSeg: Closed-Form Solution for Class-Incremental Semantic Segmentation of 2D Images and 3D Point Clouds

• URL: http://arxiv.org/abs/2412.10834v2
• Date: Sat, 12 Apr 2025 12:04:49 GMT
• Title: CFSSeg: Closed-Form Solution for Class-Incremental Semantic Segmentation of 2D Images and 3D Point Clouds
• Authors: Jiaxu Li, Rui Li, Jianyu Qi, Songning Lai, Linpu Lv, Kejia Fan, Jianheng Tang, Yutao Yue, Dongzhan Zhou, Yuanhuai Liu, Huiping Zhuang,
• Abstract summary: Class-incremental semantic segmentation (CSS) requires incrementally learning new semantic categories while retaining prior knowledge.<n>We propose CFSSeg, a novel exemplar-free approach that leverages a closed-form solution.<n>This eliminates the need for iterative gradient-based optimization and storage of past data, requiring only a single pass through new samples per step.
• Score: 9.765104818970277
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 2D images and 3D point clouds are foundational data types for multimedia applications, including real-time video analysis, augmented reality (AR), and 3D scene understanding. Class-incremental semantic segmentation (CSS) requires incrementally learning new semantic categories while retaining prior knowledge. Existing methods typically rely on computationally expensive training based on stochastic gradient descent, employing complex regularization or exemplar replay. However, stochastic gradient descent-based approaches inevitably update the model's weights for past knowledge, leading to catastrophic forgetting, a problem exacerbated by pixel/point-level granularity. To address these challenges, we propose CFSSeg, a novel exemplar-free approach that leverages a closed-form solution, offering a practical and theoretically grounded solution for continual semantic segmentation tasks. This eliminates the need for iterative gradient-based optimization and storage of past data, requiring only a single pass through new samples per step. It not only enhances computational efficiency but also provides a practical solution for dynamic, privacy-sensitive multimedia environments. Extensive experiments on 2D and 3D benchmark datasets such as Pascal VOC2012, S3DIS, and ScanNet demonstrate CFSSeg's superior performance.

Related papers

• Boosting the Class-Incremental Learning in 3D Point Clouds via Zero-Collection-Cost Basic Shape Pre-Training [12.652126803856065]
  Existing class-incremental learning methods in 3D point clouds rely on exemplars to resist forgetting of models. For exemplar-free incremental learning, the pre-trained model methods have achieved state-of-the-art results in 2D domains. This paper proposes a framework embedded with 3D geometry knowledge for incremental learning in point clouds.
  arXiv  Detail & Related papers  (2025-04-11T10:18:35Z)
• Foundation Model-Powered 3D Few-Shot Class Incremental Learning via Training-free Adaptor [9.54964908165465]
  This paper introduces a new method to tackle the Few-Shot Continual Incremental Learning problem in 3D point cloud environments. We leverage a foundational 3D model trained extensively on point cloud data. Our approach uses a dual cache system: first, it uses previous test samples based on how confident the model was in its predictions to prevent forgetting, and second, it includes a small number of new task samples to prevent overfitting.
  arXiv  Detail & Related papers  (2024-10-11T20:23:00Z)
• Temporal-Difference Variational Continual Learning [89.32940051152782]
  A crucial capability of Machine Learning models in real-world applications is the ability to continuously learn new tasks. In Continual Learning settings, models often struggle to balance learning new tasks with retaining previous knowledge. We propose new learning objectives that integrate the regularization effects of multiple previous posterior estimations.
  arXiv  Detail & Related papers  (2024-10-10T10:58:41Z)
• SLCA++: Unleash the Power of Sequential Fine-tuning for Continual Learning with Pre-training [68.7896349660824]
  We present an in-depth analysis of the progressive overfitting problem from the lens of Seq FT. Considering that the overly fast representation learning and the biased classification layer constitute this particular problem, we introduce the advanced Slow Learner with Alignment (S++) framework. Our approach involves a Slow Learner to selectively reduce the learning rate of backbone parameters, and a Alignment to align the disjoint classification layers in a post-hoc fashion.
  arXiv  Detail & Related papers  (2024-08-15T17:50:07Z)
• Augmented Efficiency: Reducing Memory Footprint and Accelerating Inference for 3D Semantic Segmentation through Hybrid Vision [9.96433151449016]
  This paper introduces a novel approach to 3D semantic segmentation, distinguished by incorporating a hybrid blend of 2D and 3D computer vision techniques. We conduct 2D semantic segmentation on RGB images linked to 3D point clouds and extend the results to 3D using an extrusion technique for specific class labels. This model serves as the current state-of-the-art 3D semantic segmentation model on the KITTI-360 dataset.
  arXiv  Detail & Related papers  (2024-07-23T00:04:10Z)
• Improving Data-aware and Parameter-aware Robustness for Continual Learning [3.480626767752489]
  This paper analyzes that this insufficiency arises from the ineffective handling of outliers. We propose a Robust Continual Learning (RCL) method to address this issue. The proposed method effectively maintains robustness and achieves new state-of-the-art (SOTA) results.
  arXiv  Detail & Related papers  (2024-05-27T11:21:26Z)
• RESSCAL3D: Resolution Scalable 3D Semantic Segmentation of Point Clouds [7.453881927237143]
  In this work, we introduce a novel architecture, dubbed RESSCAL3D, providing resolution-scalable 3D semantic segmentation of point clouds. Once a low-resolution version of the input point cloud is available, first semantic predictions can be generated in an extremely fast manner. Our experiments show that RESSCAL3D is 31-62% faster than the non-scalable baseline.
  arXiv  Detail & Related papers  (2024-04-10T09:40:56Z)
• ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning [54.68180752416519]
  Panoptic segmentation is a cutting-edge computer vision task. We introduce a novel and efficient method for continual panoptic segmentation based on Visual Prompt Tuning, dubbed ECLIPSE. Our approach involves freezing the base model parameters and fine-tuning only a small set of prompt embeddings, addressing both catastrophic forgetting and plasticity.
  arXiv  Detail & Related papers  (2024-03-29T11:31:12Z)
• Branch-Tuning: Balancing Stability and Plasticity for Continual Self-Supervised Learning [33.560003528712414]
  Self-supervised learning (SSL) has emerged as an effective paradigm for deriving general representations from vast amounts of unlabeled data. This poses a challenge in striking a balance between stability and plasticity when adapting to new information. We propose Branch-tuning, an efficient and straightforward method that achieves a balance between stability and plasticity in continual SSL.
  arXiv  Detail & Related papers  (2024-03-27T05:38:48Z)
• Rethinking Few-shot 3D Point Cloud Semantic Segmentation [62.80639841429669]
  This paper revisits few-shot 3D point cloud semantic segmentation (FS-PCS) We focus on two significant issues in the state-of-the-art: foreground leakage and sparse point distribution. To address these issues, we introduce a standardized FS-PCS setting, upon which a new benchmark is built.
  arXiv  Detail & Related papers  (2024-03-01T15:14:47Z)
• Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embedding [55.107555305760954]
  We propose a conceptually simple yet effective method that attributes forgetting to layer-wise parameter overwriting and the resulting decision boundary distortion. Our method achieves competitive accuracy performance, even with absolute superiority of zero exemplar buffer and 1.02x the base model.
  arXiv  Detail & Related papers  (2024-01-17T09:01:29Z)
• Class-Imbalanced Semi-Supervised Learning for Large-Scale Point Cloud Semantic Segmentation via Decoupling Optimization [64.36097398869774]
  Semi-supervised learning (SSL) has been an active research topic for large-scale 3D scene understanding. The existing SSL-based methods suffer from severe training bias due to class imbalance and long-tail distributions of the point cloud data. We introduce a new decoupling optimization framework, which disentangles feature representation learning and classifier in an alternative optimization manner to shift the bias decision boundary effectively.
  arXiv  Detail & Related papers  (2024-01-13T04:16:40Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• New metrics for analyzing continual learners [27.868967961503962]
  Continual Learning (CL) poses challenges to standard learning algorithms. This stability-plasticity dilemma remains central to CL and multiple metrics have been proposed to adequately measure stability and plasticity separately. We propose new metrics that account for the task's increasing difficulty.
  arXiv  Detail & Related papers  (2023-09-01T13:53:33Z)
• Clustering based Point Cloud Representation Learning for 3D Analysis [80.88995099442374]
  We propose a clustering based supervised learning scheme for point cloud analysis. Unlike current de-facto, scene-wise training paradigm, our algorithm conducts within-class clustering on the point embedding space. Our algorithm shows notable improvements on famous point cloud segmentation datasets.
  arXiv  Detail & Related papers  (2023-07-27T03:42:12Z)
• Few-Shot 3D Point Cloud Semantic Segmentation via Stratified Class-Specific Attention Based Transformer Network [22.9434434107516]
  We develop a new multi-layer transformer network for few-shot point cloud semantic segmentation. Our method achieves the new state-of-the-art performance, with 15% less inference time, over existing few-shot 3D point cloud segmentation models.
  arXiv  Detail & Related papers  (2023-03-28T00:27:54Z)
• Mitigating Forgetting in Online Continual Learning via Contrasting Semantically Distinct Augmentations [22.289830907729705]
  Online continual learning (OCL) aims to enable model learning from a non-stationary data stream to continuously acquire new knowledge as well as retain the learnt one. Main challenge comes from the "catastrophic forgetting" issue -- the inability to well remember the learnt knowledge while learning the new ones.
  arXiv  Detail & Related papers  (2022-11-10T05:29:43Z)
• MaxMatch: Semi-Supervised Learning with Worst-Case Consistency [149.03760479533855]
  We propose a worst-case consistency regularization technique for semi-supervised learning (SSL) We present a generalization bound for SSL consisting of the empirical loss terms observed on labeled and unlabeled training data separately. Motivated by this bound, we derive an SSL objective that minimizes the largest inconsistency between an original unlabeled sample and its multiple augmented variants.
  arXiv  Detail & Related papers  (2022-09-26T12:04:49Z)
• Open-Set Semi-Supervised Learning for 3D Point Cloud Understanding [62.17020485045456]
  It is commonly assumed in semi-supervised learning (SSL) that the unlabeled data are drawn from the same distribution as that of the labeled ones. We propose to selectively utilize unlabeled data through sample weighting, so that only conducive unlabeled data would be prioritized.
  arXiv  Detail & Related papers  (2022-05-02T16:09:17Z)
• Static-Dynamic Co-Teaching for Class-Incremental 3D Object Detection [71.18882803642526]
  Deep learning approaches have shown remarkable performance in the 3D object detection task. They suffer from a catastrophic performance drop when incrementally learning new classes without revisiting the old data. This "catastrophic forgetting" phenomenon impedes the deployment of 3D object detection approaches in real-world scenarios. We present the first solution - SDCoT, a novel static-dynamic co-teaching method.
  arXiv  Detail & Related papers  (2021-12-14T09:03:41Z)
• SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning [19.152041362805985]
  We consider a class-incremental semantic segmentation (CISS) problem. We propose a new method, dubbed as SSUL-M (Semantic with Unknown Label with Memory), by carefully combining several techniques tailored for semantic segmentation. We show our method achieves significantly better performance than the recent state-of-the-art baselines on the standard benchmark datasets.
  arXiv  Detail & Related papers  (2021-06-22T06:40:26Z)
• 3DSSD: Point-based 3D Single Stage Object Detector [61.67928229961813]
  We present a point-based 3D single stage object detector, named 3DSSD, achieving a good balance between accuracy and efficiency. Our method outperforms all state-of-the-art voxel-based single stage methods by a large margin, and has comparable performance to two stage point-based methods as well.
  arXiv  Detail & Related papers  (2020-02-24T12:01:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.