PILOT: A Data-Free Continual Learning Approach for Real-Time Semantic Segmentation via Boundary Guidance

• URL: http://arxiv.org/abs/2605.27128v1
• Date: Tue, 26 May 2026 15:00:12 GMT
• Title: PILOT: A Data-Free Continual Learning Approach for Real-Time Semantic Segmentation via Boundary Guidance
• Abstract summary: We propose a novel continual learning framework tailored for PIDNet.<n> PILOT allows the model to adapt to new semantic categories while preserving the knowledge of previously learned classes.<n>Results demonstrate that our approach successfully segments new classes while maintaining high mean Intersection over Union (mIoU) on the original base classes.
• Score: 4.596496450055492
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-time semantic segmentation models offer an excellent balance between accuracy and inference speed. However, deploying these models in dynamic real world environments often requires the ability to learn novel classes incrementally without retraining on the entire dataset. This capability is known as continual learning. In this regard, the standard fine-tuning methods in deep learning often fail due to catastrophic forgetting, where the model learns new information but forgets previously trained and learned classes. Contributing to this crucial domain, the current paper proposes a novel continual learning framework tailored for PIDNet, which is a widely cited state-of-the-art real-time semantic segmentation model. Our method, PILOT(Parallel Incremental Learning Over Time), introduces a real-time and lightweight strategy by implementing a parallel Derivative-branch (D-branch) designed to capture the high frequency boundary information of novel classes while freezing the trained parameters of the original segmentation network. This novel setup allows the model to adapt to new semantic categories while preserving the knowledge of previously learned classes. By using only data associated with the new class, our model significantly reduces training overhead. Experimental results demonstrate that our approach successfully segments new classes while maintaining high mean Intersection over Union (mIoU) on the original base classes, thereby comfortably outperforming all major continual learning approaches in this domain. Overall, PILOT is shown to effectively mitigate catastrophic forgetting with minimal impact on inference latency, thus maintaining real-time performance.

Related papers

• Miles: Metric Learning with Expandable Subspace for Pre-Trained Model-Based Class-Incremental Learning [57.237835833636154]
  We propose MetrIc Learning with Expandable Subspace (Miles) to harness the prior information within pre-trained knowledge.<n>Miles achieves state-of-the-art performance in various CIL settings.
  arXiv  Detail & Related papers  (2026-07-20T06:22:02Z)
• Bagging-Based Model Merging for Robust General Text Embeddings [73.51674133699196]
  General-purpose text embedding models underpin a wide range of NLP and information retrieval applications.<n>We present a systematic study of multi-task training for text embeddings from two perspectives: data scheduling and model merging.<n>We propose Bagging-based rObust mOdel Merging (BOOM), which trains multiple embedding models on sampled subsets and merges them into a single model.
  arXiv  Detail & Related papers  (2026-02-05T15:45:08Z)
• Entropy-Guided Token Dropout: Training Autoregressive Language Models with Limited Domain Data [89.96277093034547]
  We introduce EntroDrop, an entropy-guided token dropout method that functions as structured data regularization.<n>We show that EntroDrop consistently outperforms standard regularization baselines and maintains robust performance throughout extended multi-epoch training.
  arXiv  Detail & Related papers  (2025-12-29T12:35:51Z)
• Forgetting, Ignorance or Myopia: Revisiting Key Challenges in Online Continual Learning [29.65600202138321]
  In high-speed data stream environments, data do not pause to accommodate slow models. Model's ignorance: the single-pass nature of OCL challenges models to learn effective features within constrained training time. Model's myopia: the local learning nature of OCL leads the model to adopt overly simplified, task-specific features.
  arXiv  Detail & Related papers  (2024-09-28T05:24:56Z)
• ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning [54.68180752416519]
  Panoptic segmentation is a cutting-edge computer vision task. We introduce a novel and efficient method for continual panoptic segmentation based on Visual Prompt Tuning, dubbed ECLIPSE. Our approach involves freezing the base model parameters and fine-tuning only a small set of prompt embeddings, addressing both catastrophic forgetting and plasticity.
  arXiv  Detail & Related papers  (2024-03-29T11:31:12Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• PILOT: A Pre-Trained Model-Based Continual Learning Toolbox [65.57123249246358]
  This paper introduces a pre-trained model-based continual learning toolbox known as PILOT.<n>On the one hand, PILOT implements some state-of-the-art class-incremental learning algorithms based on pre-trained models, such as L2P, DualPrompt, and CODA-Prompt.<n>On the other hand, PILOT fits typical class-incremental learning algorithms within the context of pre-trained models to evaluate their effectiveness.
  arXiv  Detail & Related papers  (2023-09-13T17:55:11Z)
• Complementary Learning Subnetworks for Parameter-Efficient Class-Incremental Learning [40.13416912075668]
  We propose a rehearsal-free CIL approach that learns continually via the synergy between two Complementary Learning Subnetworks. Our method achieves competitive results against state-of-the-art methods, especially in accuracy gain, memory cost, training efficiency, and task-order.
  arXiv  Detail & Related papers  (2023-06-21T01:43:25Z)
• Hyperparameter-free Continuous Learning for Domain Classification in Natural Language Understanding [60.226644697970116]
  Domain classification is the fundamental task in natural language understanding (NLU) Most existing continual learning approaches suffer from low accuracy and performance fluctuation. We propose a hyper parameter-free continual learning model for text data that can stably produce high performance under various environments.
  arXiv  Detail & Related papers  (2022-01-05T02:46:16Z)
• An EM Framework for Online Incremental Learning of Semantic Segmentation [37.94734474090863]
  We propose an incremental learning strategy that can adapt deep segmentation models without catastrophic forgetting, using a streaming input data with pixel annotations on the novel classes only. We validate our approach on the PASCAL VOC 2012 and ADE20K datasets, and the results demonstrate its superior performance over the existing incremental methods.
  arXiv  Detail & Related papers  (2021-08-08T11:30:09Z)
• Modeling the Background for Incremental Learning in Semantic Segmentation [39.025848280224785]
  Deep architectures are vulnerable to catastrophic forgetting. This paper addresses this problem in the context of semantic segmentation. We propose a new distillation-based framework which explicitly accounts for this shift.
  arXiv  Detail & Related papers  (2020-02-03T13:30:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.