Hierarchical Dual-Subspace Decoupling for Continual Learning in Vision-Language Models

• URL: http://arxiv.org/abs/2605.07512v1
• Date: Fri, 08 May 2026 09:42:05 GMT
• Title: Hierarchical Dual-Subspace Decoupling for Continual Learning in Vision-Language Models
• Abstract summary: Class-incremental learning aims to continuously acquire new knowledge while preserving previously learned information, thereby mitigating catastrophic forgetting.<n>We propose HDSD, a Hierarchical Dual-Subspace Decoupling framework for continual learning in vision-language models.<n> Specifically, we introduce a lightweight Feature Modulation Module (FMM) that explicitly decomposes the parameter space into general and task-specific subspaces.
• Score: 63.169731309799324
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Class-incremental learning aims to continuously acquire new knowledge while preserving previously learned information, thereby mitigating catastrophic forgetting. Existing methods primarily restrict parameter updates but often overlook their structural properties in high-dimensional spaces. From a subspace perspective, updates induced by different tasks tend to lie in multiple overlapping low-rank subspaces, leading to cross-task subspace interference and severe forgetting. To address this issue, we propose HDSD, a Hierarchical Dual-Subspace Decoupling framework for continual learning in vision-language models. Specifically, we introduce a lightweight Feature Modulation Module (FMM) that explicitly decomposes the parameter space into general and task-specific subspaces. Building on this design, we develop two complementary components. First, a General Fusion Module (GFM) evaluates relative parameter changes across tasks and uses an adaptive threshold to capture stable and transferable knowledge. Second, a Hierarchical Learning Module (HLM) performs structured parameter decomposition via Singular Value Decomposition (SVD) and uses a scaling mechanism to constrain updates within distinct subspace scales. Together, these designs reduce subspace interference and parameter drift. Extensive experiments on conventional benchmarks show that HDSD achieves state-of-the-art results.

Related papers

• Miles: Metric Learning with Expandable Subspace for Pre-Trained Model-Based Class-Incremental Learning [57.237835833636154]
  We propose MetrIc Learning with Expandable Subspace (Miles) to harness the prior information within pre-trained knowledge.<n>Miles achieves state-of-the-art performance in various CIL settings.
  arXiv  Detail & Related papers  (2026-07-20T06:22:02Z)
• KeepLoRA++: Continual Learning with Layer-Scaled Residual Gradient Adaptation [67.23448686598414]
  Continual learning for pre-trained vision-language models requires balancing three competing objectives.<n>This paper presents KeepLoRA++, balancing these objectives through a unified dual-dimensional knowledge retention mechanism.
  arXiv  Detail & Related papers  (2026-06-15T05:57:59Z)
• iGSP:Implicit Gradient Subspace Projection for Efficient Continual Learning of Vision-Language Models [14.623213614166259]
  iGSP is a novel framework that achieves efficient adaptation via implicit gradient subspace projection.<n>Experiments on the MTIL benchmark demonstrate that iGSP achieves state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2026-05-19T03:22:14Z)
• On Catastrophic Forgetting in Low-Rank Decomposition-Based Parameter-Efficient Fine-Tuning [14.815508281465272]
  We show that forgetting is strongly influenced by the geometry and parameterization of the update subspace.<n>Our findings highlight update subspace design as a key factor in continual learning and offer practical guidance for selecting efficient adaptation strategies in sequential settings.
  arXiv  Detail & Related papers  (2026-03-10T13:53:25Z)
• Model Merging in the Essential Subspace [78.5390284258307]
  Model merging aims to integrate multiple task-specific fine-tuned models into a single multi-task model without additional training.<n>Despite extensive research, task interference remains a major obstacle that often undermines the performance of merged models.<n>We propose ESM (Essential Subspace Merging), a robust framework for effective model merging.
  arXiv  Detail & Related papers  (2026-02-23T00:33:38Z)
• Semantic-Guided Dynamic Sparsification for Pre-Trained Model-based Class-Incremental Learning [25.808884795490613]
  Class-Incremental Learning (CIL) requires a model to continually learn new classes without forgetting old ones.<n>We argue that this parameter-constraining method is detrimental to plasticity.<n>We propose Semantic-Guided Dynamic Sparsification (SGDS), a novel method that proactively guides the activation space.
  arXiv  Detail & Related papers  (2026-01-29T07:15:05Z)
• Unifying Search and Recommendation in LLMs via Gradient Multi-Subspace Tuning [33.69176756907003]
  Gradient Multi-Subspace Tuning (GEMS) is a novel framework that unifies search and recommendation tasks.<n>We show that GEMS consistently outperforms the state-of-the-art baselines across both search and recommendation tasks.
  arXiv  Detail & Related papers  (2026-01-14T14:03:07Z)
• Efficient Multi-Instance Generation with Janus-Pro-Dirven Prompt Parsing [53.295515505026096]
  Janus-Pro-driven Prompt Parsing is a prompt- parsing module that bridges text understanding and layout generation.<n>MIGLoRA is a parameter-efficient plug-in integrating Low-Rank Adaptation into UNet (SD1.5) and DiT (SD3) backbones.<n>The proposed method achieves state-of-the-art performance on COCO and LVIS benchmarks while maintaining parameter efficiency.
  arXiv  Detail & Related papers  (2025-03-27T00:59:14Z)
• Learning and Evaluating Hierarchical Feature Representations [3.770103075126785]
  We propose a novel framework, Hierarchical Composition of Orthogonal Subspaces (Hier-COS)<n>Hier-COS learns to map deep feature embeddings into a vector space that is, by design, consistent with the structure of a given taxonomy tree.<n>We demonstrate that Hier-COS achieves state-of-the-art hierarchical performance across all the datasets while simultaneously beating top-1 accuracy in all but one case.
  arXiv  Detail & Related papers  (2025-03-10T20:59:41Z)
• Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning [115.79349923044663]
  Few-shot class-incremental learning (FSCIL) aims to incrementally learn novel classes from limited examples.<n>Existing methods face a critical dilemma: static architectures rely on a fixed parameter space to learn from data that arrive sequentially, prone to overfitting to the current session.<n>In this study, we explore the potential of Selective State Space Models (SSMs) for FSCIL.
  arXiv  Detail & Related papers  (2024-07-08T17:09:39Z)
• Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning [65.57123249246358]
  We propose ExpAndable Subspace Ensemble (EASE) for PTM-based CIL. We train a distinct lightweight adapter module for each new task, aiming to create task-specific subspaces. Our prototype complement strategy synthesizes old classes' new features without using any old class instance.
  arXiv  Detail & Related papers  (2024-03-18T17:58:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.