Evaluating Continual Test-Time Adaptation for Contextual and Semantic Domain Shifts

• URL: http://arxiv.org/abs/2208.08767v1
• Date: Thu, 18 Aug 2022 11:05:55 GMT
• Title: Evaluating Continual Test-Time Adaptation for Contextual and Semantic Domain Shifts
• Authors: Tommie Kerssies, Joaquin Vanschoren and Mert K{\i}l{\i}\c{c}kaya
• Abstract summary: We adapt a pre-trained Convolutional Neural Network to domain shifts at test time. We evaluate the state of the art on two realistic and challenging sources of domain shifts, namely contextual and semantic shifts. Test-time adaptation methods perform better and forget less on contextual shifts compared to semantic shifts.
• Score: 3.4161707164978137
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, our goal is to adapt a pre-trained Convolutional Neural Network to domain shifts at test time. We do so continually with the incoming stream of test batches, without labels. Existing literature mostly operates on artificial shifts obtained via adversarial perturbations of a test image. Motivated by this, we evaluate the state of the art on two realistic and challenging sources of domain shifts, namely contextual and semantic shifts. Contextual shifts correspond to the environment types, for example a model pre-trained on indoor context has to adapt to the outdoor context on CORe-50 [7]. Semantic shifts correspond to the capture types, for example a model pre-trained on natural images has to adapt to cliparts, sketches and paintings on DomainNet [10]. We include in our analysis recent techniques such as Prediction-Time Batch Normalization (BN) [8], Test Entropy Minimization (TENT) [16] and Continual Test-Time Adaptation (CoTTA) [17]. Our findings are three-fold: i) Test-time adaptation methods perform better and forget less on contextual shifts compared to semantic shifts, ii) TENT outperforms other methods on short-term adaptation, whereas CoTTA outpeforms other methods on long-term adaptation, iii) BN is most reliable and robust.

Related papers

• Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Models [19.683461002518147]
  Test-Time Prototype Shifting (TPS) is a pioneering approach designed to adapt vision-language models to test datasets using unlabeled test inputs. TPS not only facilitates optimization-free prototype reuse for subsequent predictions but also enables seamless integration with current advancements in prompt engineering. A notable aspect of our framework is its significantly reduced memory and computational demands when compared to conventional text-prompt tuning methods.
  arXiv  Detail & Related papers  (2024-03-19T17:54:34Z)
• On Pitfalls of Test-Time Adaptation [82.8392232222119]
  Test-Time Adaptation (TTA) has emerged as a promising approach for tackling the robustness challenge under distribution shifts. We present TTAB, a test-time adaptation benchmark that encompasses ten state-of-the-art algorithms, a diverse array of distribution shifts, and two evaluation protocols.
  arXiv  Detail & Related papers  (2023-06-06T09:35:29Z)
• Robust Mean Teacher for Continual and Gradual Test-Time Adaptation [5.744133015573047]
  Gradual test-time adaptation (TTA) considers not only a single domain shift, but a sequence of shifts. We propose and show that in the setting of TTA, the symmetric cross-entropy is better suited as a consistency loss for mean teachers. We demonstrate the effectiveness of our proposed method 'robust mean teacher' (RMT) on the continual and gradual corruption benchmarks.
  arXiv  Detail & Related papers  (2022-11-23T16:14:45Z)
• TeST: Test-time Self-Training under Distribution Shift [99.68465267994783]
  Test-Time Self-Training (TeST) is a technique that takes as input a model trained on some source data and a novel data distribution at test time. We find that models adapted using TeST significantly improve over baseline test-time adaptation algorithms.
  arXiv  Detail & Related papers  (2022-09-23T07:47:33Z)
• Domain Adaptation with Adversarial Training on Penultimate Activations [82.9977759320565]
  Enhancing model prediction confidence on unlabeled target data is an important objective in Unsupervised Domain Adaptation (UDA) We show that this strategy is more efficient and better correlated with the objective of boosting prediction confidence than adversarial training on input images or intermediate features.
  arXiv  Detail & Related papers  (2022-08-26T19:50:46Z)
• Gradual Test-Time Adaptation by Self-Training and Style Transfer [5.110894308882439]
  We show the natural connection between gradual domain adaptation and test-time adaptation. We propose a new method based on self-training and style transfer. We show the effectiveness of our method on the continual and gradual CIFAR10C, CIFAR100C, and ImageNet-C benchmark.
  arXiv  Detail & Related papers  (2022-08-16T13:12:19Z)
• Test-time Batch Normalization [61.292862024903584]
  Deep neural networks often suffer the data distribution shift between training and testing. We revisit the batch normalization (BN) in the training process and reveal two key insights benefiting test-time optimization. We propose a novel test-time BN layer design, GpreBN, which is optimized during testing by minimizing Entropy loss.
  arXiv  Detail & Related papers  (2022-05-20T14:33:39Z)
• Test time Adaptation through Perturbation Robustness [1.52292571922932]
  We tackle the problem of adapting to domain shift at inference time. We do not change the training process, but quickly adapt the model at test-time to handle any domain shift. Our method is at par or significantly outperforms previous methods.
  arXiv  Detail & Related papers  (2021-10-19T20:00:58Z)
• Test-time Batch Statistics Calibration for Covariate Shift [66.7044675981449]
  We propose to adapt the deep models to the novel environment during inference. We present a general formulation $alpha$-BN to calibrate the batch statistics. We also present a novel loss function to form a unified test time adaptation framework Core.
  arXiv  Detail & Related papers  (2021-10-06T08:45:03Z)
• Adaptive Risk Minimization: Learning to Adapt to Domain Shift [109.87561509436016]
  A fundamental assumption of most machine learning algorithms is that the training and test data are drawn from the same underlying distribution. In this work, we consider the problem setting of domain generalization, where the training data are structured into domains and there may be multiple test time shifts. We introduce the framework of adaptive risk minimization (ARM), in which models are directly optimized for effective adaptation to shift by learning to adapt on the training domains.
  arXiv  Detail & Related papers  (2020-07-06T17:59:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.