Enhancing Test Time Adaptation with Few-shot Guidance

• URL: http://arxiv.org/abs/2409.01341v1
• Date: Mon, 2 Sep 2024 15:50:48 GMT
• Title: Enhancing Test Time Adaptation with Few-shot Guidance
• Authors: Siqi Luo, Yi Xin, Yuntao Du, Zhongwei Wan, Tao Tan, Guangtao Zhai, Xiaohong Liu,
• Abstract summary: Deep neural networks often encounter significant performance drops while facing with domain shifts between training (source) and test (target) data. Test Time Adaptation (TTA) methods have been proposed to adapt pre-trained source model to handle out-of-distribution streaming target data. We develop Few-Shot Test Time Adaptation (FS-TTA), a novel and practical setting that utilizes a few-shot support set on top of TTA.
• Score: 35.13317598777832
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep neural networks often encounter significant performance drops while facing with domain shifts between training (source) and test (target) data. To address this issue, Test Time Adaptation (TTA) methods have been proposed to adapt pre-trained source model to handle out-of-distribution streaming target data. Although these methods offer some relief, they lack a reliable mechanism for domain shift correction, which can often be erratic in real-world applications. In response, we develop Few-Shot Test Time Adaptation (FS-TTA), a novel and practical setting that utilizes a few-shot support set on top of TTA. Adhering to the principle of few inputs, big gains, FS-TTA reduces blind exploration in unseen target domains. Furthermore, we propose a two-stage framework to tackle FS-TTA, including (i) fine-tuning the pre-trained source model with few-shot support set, along with using feature diversity augmentation module to avoid overfitting, (ii) implementing test time adaptation based on prototype memory bank guidance to produce high quality pseudo-label for model adaptation. Through extensive experiments on three cross-domain classification benchmarks, we demonstrate the superior performance and reliability of our FS-TTA and framework.

Related papers

• Bi-TTA: Bidirectional Test-Time Adapter for Remote Physiological Measurement [25.11883761217408]
  Remote photoplethysmography (r) is gaining prominence for its non-invasive approach to monitoring physiological signals using only cameras. Despite its promise, the adaptability of r models to new domains is hindered due to the environmental sensitivity of physiological signals. We present Bi-TTA, a novel expert knowledge-based Bidirectional Test-Time Adapter framework.
  arXiv  Detail & Related papers  (2024-09-25T19:55:20Z)
• Low Saturation Confidence Distribution-based Test-Time Adaptation for Cross-Domain Remote Sensing Image Classification [3.79505282305064]
  We propose a novel and comprehensive test time adaptation (TTA) method -- Low Saturation Confidence Distribution Test Time Adaptation (LSCD-TTA) LSCD-TTA specifically considers the distribution characteristics of remote sensing images, including three main parts that concentrate on different optimization directions. The experimental results show that LSCD-TTA achieves a significant gain of 4.96%-10.51% with Resnet-50 and 5.33%-12.49% with Resnet-101 in average accuracy compared to other state-of-the-art DA and TTA methods.
  arXiv  Detail & Related papers  (2024-08-29T05:04:25Z)
• Test-Time Model Adaptation with Only Forward Passes [68.11784295706995]
  Test-time adaptation has proven effective in adapting a given trained model to unseen test samples with potential distribution shifts. We propose a test-time Forward-Optimization Adaptation (FOA) method. FOA runs on quantized 8-bit ViT, outperforms gradient-based TENT on full-precision 32-bit ViT, and achieves an up to 24-fold memory reduction on ImageNet-C.
  arXiv  Detail & Related papers  (2024-04-02T05:34:33Z)
• Test-Time Domain Generalization for Face Anti-Spoofing [60.94384914275116]
  Face Anti-Spoofing (FAS) is pivotal in safeguarding facial recognition systems against presentation attacks. We introduce a novel Test-Time Domain Generalization framework for FAS, which leverages the testing data to boost the model's generalizability. Our method, consisting of Test-Time Style Projection (TTSP) and Diverse Style Shifts Simulation (DSSS), effectively projects the unseen data to the seen domain space.
  arXiv  Detail & Related papers  (2024-03-28T11:50:23Z)
• AR-TTA: A Simple Method for Real-World Continual Test-Time Adaptation [1.4530711901349282]
  We propose to validate test-time adaptation methods using datasets for autonomous driving, namely CLAD-C and SHIFT. We observe that current test-time adaptation methods struggle to effectively handle varying degrees of domain shift. We enhance the well-established self-training framework by incorporating a small memory buffer to increase model stability.
  arXiv  Detail & Related papers  (2023-09-18T19:34:23Z)
• Consistency Regularization for Generalizable Source-free Domain Adaptation [62.654883736925456]
  Source-free domain adaptation (SFDA) aims to adapt a well-trained source model to an unlabelled target domain without accessing the source dataset. Existing SFDA methods ONLY assess their adapted models on the target training set, neglecting the data from unseen but identically distributed testing sets. We propose a consistency regularization framework to develop a more generalizable SFDA method.
  arXiv  Detail & Related papers  (2023-08-03T07:45:53Z)
• Improved Test-Time Adaptation for Domain Generalization [48.239665441875374]
  Test-time training (TTT) adapts the learned model with test data. This work addresses two main factors: selecting an appropriate auxiliary TTT task for updating and identifying reliable parameters to update during the test phase. We introduce additional adaptive parameters for the trained model, and we suggest only updating the adaptive parameters during the test phase.
  arXiv  Detail & Related papers  (2023-04-10T10:12:38Z)
• CAFA: Class-Aware Feature Alignment for Test-Time Adaptation [50.26963784271912]
  Test-time adaptation (TTA) aims to address this challenge by adapting a model to unlabeled data at test time. We propose a simple yet effective feature alignment loss, termed as Class-Aware Feature Alignment (CAFA), which simultaneously encourages a model to learn target representations in a class-discriminative manner.
  arXiv  Detail & Related papers  (2022-06-01T03:02:07Z)
• Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognition [65.84978547406753]
  Test-time Adaptation aims to adapt the model trained on source domains to yield better predictions for test samples. Single-Utterance Test-time Adaptation (SUTA) is the first TTA study in speech area to our best knowledge.
  arXiv  Detail & Related papers  (2022-03-27T06:38:39Z)
• Adaptive Risk Minimization: Learning to Adapt to Domain Shift [109.87561509436016]
  A fundamental assumption of most machine learning algorithms is that the training and test data are drawn from the same underlying distribution. In this work, we consider the problem setting of domain generalization, where the training data are structured into domains and there may be multiple test time shifts. We introduce the framework of adaptive risk minimization (ARM), in which models are directly optimized for effective adaptation to shift by learning to adapt on the training domains.
  arXiv  Detail & Related papers  (2020-07-06T17:59:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.