Continual Learning of Domain-Invariant Representations

• URL: http://arxiv.org/abs/2605.15775v1
• Date: Fri, 15 May 2026 09:31:10 GMT
• Title: Continual Learning of Domain-Invariant Representations
• Abstract summary: We introduce a class of CL methods that sequentially learn representations capturing invariant structures across domains.<n>Our methods consistently outperform existing CL baselines in terms of generalization to unseen target domains.
• Score: 32.573842841321316
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Continual learning (CL) aims to train models sequentially over multiple domains without forgetting previously learned knowledge. However, existing CL methods optimize for in-domain performance and are therefore prone to learning spurious, domain-specific cues (``shortcut learning''), which limits generalization to unseen domains after deployment. In this paper, we address this limitation through continual learning of domain-invariant representation. We introduce a broad class of CL methods that sequentially learn representations capturing invariant structures across domains. Our methods are motivated by the observation that such invariant structures often preserve the underlying causal mechanisms, which can reduce the risk of overfitting to domain-specific cues and thus offer better out-of-domain generalization. Our proposed CL methods combine replay-based training with a tailored sequential invariance alignment to learn -- and preserve -- invariant structures over time. We evaluate our methods under a deployment-oriented protocol that measures performance on unseen target domains. Across six benchmark and real-world datasets spanning vision, medicine, manufacturing, and ecology, our methods consistently outperform existing CL baselines in terms of generalization to unseen target domains. As an ablation, we further show that naïve extensions of sequential training with existing domain-invariant representation learning (DIRL) methods provide only limited benefits. To the best of our knowledge, this is the first work to develop domain-invariant representation methods for CL.

Related papers

• Granular Ball Guided Stable Latent Domain Discovery for Domain-General Crowd Counting [19.18297173252027]
  We propose a granular ball guided stable latent domain discovery framework for domain-general crowd counting.<n>The proposed method first groups samples into compact local granular balls and then clusters granular ball centers as representatives to infer pseudo-domains.<n>On top of the discovered latent domains, we develop a two-branch learning framework that improves transferable semantic representations.
  arXiv  Detail & Related papers  (2026-03-25T09:12:35Z)
• Free Lunch to Meet the Gap: Intermediate Domain Reconstruction for Cross-Domain Few-Shot Learning [20.048013939398484]
  Cross-Domain Few-Shot Learning endeavors to transfer generalized knowledge from the source domain to target domains.<n>We make novel attempts to construct Intermediate Domain Proxies (IDP) with source feature embeddings as the codebook.<n>We develop a fast domain alignment method to use these proxies as learning guidance for target domain feature transformation.
  arXiv  Detail & Related papers  (2025-11-18T09:14:06Z)
• LLM-RecG: A Semantic Bias-Aware Framework for Zero-Shot Sequential Recommendation [5.512301280728178]
  Zero-shot cross-domain sequential recommendation (ZCDSR) enables predictions in unseen domains without additional training or fine-tuning.<n>Recent advancements in large language models (LLMs) have significantly enhanced ZCDSR by facilitating cross-domain knowledge transfer.<n>We propose a novel semantic bias-aware framework that improves cross-domain alignment at both the item and sequential levels.
  arXiv  Detail & Related papers  (2025-01-31T15:43:21Z)
• In the Era of Prompt Learning with Vision-Language Models [1.060608983034705]
  We introduce textscStyLIP, a novel domain-agnostic prompt learning strategy for Domain Generalization (DG) StyLIP disentangles visual style and content in CLIPs vision encoder by using style projectors to learn domain-specific prompt tokens. We also propose AD-CLIP for unsupervised domain adaptation (DA), leveraging CLIPs frozen vision backbone.
  arXiv  Detail & Related papers  (2024-11-07T17:31:21Z)
• Rethinking Domain Adaptation and Generalization in the Era of CLIP [27.12334798260904]
  We show that a simple domain prior boosts CLIP's zero-shot recognition in a specific domain. We also create a benchmark for zero-shot adaptation and pseudo-labeling based self-training with CLIP. We propose to improve the task generalization ability of CLIP from multiple unlabeled domains.
  arXiv  Detail & Related papers  (2024-07-21T14:09:14Z)
• Memory-Efficient Prompt Tuning for Incremental Histopathology Classification [69.46798702300042]
  We present a memory-efficient prompt tuning framework to cultivate model generalization potential in economical memory cost. We have extensively evaluated our framework with two histopathology tasks, i.e., breast cancer metastasis classification and epithelium-stroma tissue classification.
  arXiv  Detail & Related papers  (2024-01-22T03:24:45Z)
• MoP-CLIP: A Mixture of Prompt-Tuned CLIP Models for Domain Incremental Learning [12.737883740101438]
  We present a novel DIL approach based on a mixture of prompt-tuned CLIP models (MoP-CLIP) At the training stage we model the features distribution of every class in each domain, learning individual text and visual prompts to adapt to a given domain. At inference, the learned distributions allow us to identify whether a given test sample belongs to a known domain, selecting the correct prompt for the classification task, or from an unseen domain.
  arXiv  Detail & Related papers  (2023-07-11T18:17:50Z)
• Transferrable Contrastive Learning for Visual Domain Adaptation [108.98041306507372]
  Transferrable Contrastive Learning (TCL) is a self-supervised learning paradigm tailored for domain adaptation. TCL penalizes cross-domain intra-class domain discrepancy between source and target through a clean and novel contrastive loss. The free lunch is, thanks to the incorporation of contrastive learning, TCL relies on a moving-averaged key encoder that naturally achieves a temporally ensembled version of pseudo labels for target data.
  arXiv  Detail & Related papers  (2021-12-14T16:23:01Z)
• Structured Latent Embeddings for Recognizing Unseen Classes in Unseen Domains [108.11746235308046]
  We propose a novel approach that learns domain-agnostic structured latent embeddings by projecting images from different domains. Our experiments on the challenging DomainNet and DomainNet-LS benchmarks show the superiority of our approach over existing methods.
  arXiv  Detail & Related papers  (2021-07-12T17:57:46Z)
• AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection [90.18752912204778]
  Unsupervised domain adaptation for object detection is a challenging problem with many real-world applications. We propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain-adversarial training. Our approach significantly outperforms the state-of-the-art methods on standard benchmarks for both similar and dissimilar domain adaptations.
  arXiv  Detail & Related papers  (2021-06-10T05:01:20Z)
• Cluster, Split, Fuse, and Update: Meta-Learning for Open Compound Domain Adaptive Semantic Segmentation [102.42638795864178]
  We propose a principled meta-learning based approach to OCDA for semantic segmentation. We cluster target domain into multiple sub-target domains by image styles, extracted in an unsupervised manner. A meta-learner is thereafter deployed to learn to fuse sub-target domain-specific predictions, conditioned upon the style code. We learn to online update the model by model-agnostic meta-learning (MAML) algorithm, thus to further improve generalization.
  arXiv  Detail & Related papers  (2020-12-15T13:21:54Z)
• Class-Incremental Domain Adaptation [56.72064953133832]
  We introduce a practical Domain Adaptation (DA) paradigm called Class-Incremental Domain Adaptation (CIDA) Existing DA methods tackle domain-shift but are unsuitable for learning novel target-domain classes. Our approach yields superior performance as compared to both DA and CI methods in the CIDA paradigm.
  arXiv  Detail & Related papers  (2020-08-04T07:55:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.