Remembering by Reconstructing: Domain Incremental Learning With Test-Time Training on Video Streams

• URL: http://arxiv.org/abs/2605.31108v1
• Date: Fri, 29 May 2026 10:17:09 GMT
• Title: Remembering by Reconstructing: Domain Incremental Learning With Test-Time Training on Video Streams
• Abstract summary: We introduce a novel approach to domain incremental learning, adapting models over time to evolving, non-stationary data.<n>We do not attempt to avoid catastrophic forgetting, but rather allow it and exploit it.<n>Our scheme is especially well-suited to real-world streaming data, such as video, where consecutive samples are highly correlated.
• Score: 37.5115685757579
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work we introduce a novel approach to domain incremental learning, adapting models over time to evolving, non-stationary data. In contrast to other works, we do not attempt to avoid catastrophic forgetting, but rather allow it and exploit it. Our model combines a main task head with a self-supervised masked autoencoder (MAE) head. We then learn domain-specific LoRA adapters during incremental training. Each adapter specializes to its domain, naturally inducing forgetting on other domains in both heads. At inference, we perform online test-time training on the self-supervised MAE head to identify which LoRAs best matches the current input, so the model can `remember' the domain again. Our scheme is especially well-suited to real-world streaming data, such as video, where consecutive samples are highly correlated and domain shifts are gradual. We demonstrate our method on domain-incremental action recognition and semantic segmentation tasks.

Related papers

• Incremental Open-set Domain Adaptation [27.171935835686117]
  Catastrophic forgetting makes neural network models unstable when learning visual domains consecutively. We develop a forgetting-resistant incremental learning strategy for image classification.
  arXiv  Detail & Related papers  (2024-08-31T19:37:54Z)
• Adapting to Distribution Shift by Visual Domain Prompt Generation [34.19066857066073]
  We adapt a model at test-time using a few unlabeled data to address distribution shifts. We build a knowledge bank to learn the transferable knowledge from source domains. The proposed method outperforms previous work on 5 large-scale benchmarks including WILDS and DomainNet.
  arXiv  Detail & Related papers  (2024-05-05T02:44:04Z)
• CDFSL-V: Cross-Domain Few-Shot Learning for Videos [58.37446811360741]
  Few-shot video action recognition is an effective approach to recognizing new categories with only a few labeled examples. Existing methods in video action recognition rely on large labeled datasets from the same domain. We propose a novel cross-domain few-shot video action recognition method that leverages self-supervised learning and curriculum learning.
  arXiv  Detail & Related papers  (2023-09-07T19:44:27Z)
• Unsupervised Domain Adaptation for Anatomical Landmark Detection [5.070344284426738]
  We propose a novel framework for anatomical landmark detection under the setting of unsupervised domain adaptation (UDA) The framework leverages self-training and domain adversarial learning to address the domain gap during adaptation. Our experiments on cephalometric and lung landmark detection show the effectiveness of the method, which reduces the domain gap by a large margin and outperforms other UDA methods consistently.
  arXiv  Detail & Related papers  (2023-08-25T10:22:13Z)
• Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Experts [33.21435044949033]
  Most existing methods perform training on multiple source domains using a single model. We propose a novel framework for unsupervised test-time adaptation, which is formulated as a knowledge distillation process.
  arXiv  Detail & Related papers  (2022-10-08T02:28:10Z)
• Forget Less, Count Better: A Domain-Incremental Self-Distillation Learning Benchmark for Lifelong Crowd Counting [51.44987756859706]
  Off-the-shelf methods have some drawbacks to handle multiple domains. Lifelong Crowd Counting aims at alleviating the catastrophic forgetting and improving the generalization ability.
  arXiv  Detail & Related papers  (2022-05-06T15:37:56Z)
• Unsupervised Lifelong Person Re-identification via Contrastive Rehearsal [7.983523975392535]
  Unsupervised lifelong person ReID focuses on continuously conducting unsupervised domain adaptation on new domains. We set an image-to-image similarity constraint between old and new models to regularize the model updates in a way that suits old knowledge. Our proposed lifelong method achieves strong generalizability, which significantly outperforms previous lifelong methods on both seen and unseen domains.
  arXiv  Detail & Related papers  (2022-03-12T15:44:08Z)
• Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training [67.71228426496013]
  We show that using target domain data during pre-training leads to large performance improvements across a variety of setups. We find that pre-training on multiple domains improves performance generalization on domains not seen during training.
  arXiv  Detail & Related papers  (2021-04-02T12:53:15Z)
• Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptation [91.58443042554903]
  We propose an end-to-end Prototypical Cross-domain Self-Supervised Learning (PCS) framework for Few-shot Unsupervised Domain Adaptation (FUDA) PCS not only performs cross-domain low-level feature alignment, but it also encodes and aligns semantic structures in the shared embedding space across domains. Compared with state-of-the-art methods, PCS improves the mean classification accuracy over different domain pairs on FUDA by 10.5%, 3.5%, 9.0%, and 13.2% on Office, Office-Home, VisDA-2017, and DomainNet, respectively.
  arXiv  Detail & Related papers  (2021-03-31T02:07:42Z)
• Batch Normalization Embeddings for Deep Domain Generalization [50.51405390150066]
  Domain generalization aims at training machine learning models to perform robustly across different and unseen domains. We show a significant increase in classification accuracy over current state-of-the-art techniques on popular domain generalization benchmarks.
  arXiv  Detail & Related papers  (2020-11-25T12:02:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.