Universal Test-time Adaptation through Weight Ensembling, Diversity Weighting, and Prior Correction

• URL: http://arxiv.org/abs/2306.00650v2
• Date: Wed, 25 Oct 2023 20:50:19 GMT
• Title: Universal Test-time Adaptation through Weight Ensembling, Diversity Weighting, and Prior Correction
• Authors: Robert A. Marsden, Mario D\"obler, Bin Yang
• Abstract summary: Test-time adaptation (TTA) continues to update the model after deployment, leveraging the current test data. We identify and highlight several challenges a self-training based method has to deal with. To prevent the model from becoming biased, we leverage a dataset and model-agnostic certainty and diversity weighting.
• Score: 3.5139431332194198
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Since distribution shifts are likely to occur during test-time and can drastically decrease the model's performance, online test-time adaptation (TTA) continues to update the model after deployment, leveraging the current test data. Clearly, a method proposed for online TTA has to perform well for all kinds of environmental conditions. By introducing the variable factors domain non-stationarity and temporal correlation, we first unfold all practically relevant settings and define the entity as universal TTA. We want to highlight that this is the first work that covers such a broad spectrum, which is indispensable for the use in practice. To tackle the problem of universal TTA, we identify and highlight several challenges a self-training based method has to deal with: 1) model bias and the occurrence of trivial solutions when performing entropy minimization on varying sequence lengths with and without multiple domain shifts, 2) loss of generalization which exacerbates the adaptation to multiple domain shifts and the occurrence of catastrophic forgetting, and 3) performance degradation due to shifts in class prior. To prevent the model from becoming biased, we leverage a dataset and model-agnostic certainty and diversity weighting. In order to maintain generalization and prevent catastrophic forgetting, we propose to continually weight-average the source and adapted model. To compensate for disparities in the class prior during test-time, we propose an adaptive prior correction scheme that reweights the model's predictions. We evaluate our approach, named ROID, on a wide range of settings, datasets, and models, setting new standards in the field of universal TTA. Code is available at: https://github.com/mariodoebler/test-time-adaptation

Related papers

• Towards Real-World Test-Time Adaptation: Tri-Net Self-Training with Balanced Normalization [52.03927261909813]
  Existing works mainly consider real-world test-time adaptation under non-i.i.d. data stream and continual domain shift. We argue failure of state-of-the-art methods is first caused by indiscriminately adapting normalization layers to imbalanced testing data. The final TTA model, termed as TRIBE, is built upon a tri-net architecture with balanced batchnorm layers.
  arXiv  Detail & Related papers  (2023-09-26T14:06:26Z)
• RDumb: A simple approach that questions our progress in continual test-time adaptation [12.374649969346441]
  Test-Time Adaptation (TTA) allows to update pre-trained models to changing data distributions at deployment time. Recent work proposed and applied methods for continual adaptation over long timescales. We find that eventually all but one state-of-the-art methods collapse and perform worse than a non-adapting model.
  arXiv  Detail & Related papers  (2023-06-08T17:52:34Z)
• On Pitfalls of Test-Time Adaptation [82.8392232222119]
  Test-Time Adaptation (TTA) has emerged as a promising approach for tackling the robustness challenge under distribution shifts. We present TTAB, a test-time adaptation benchmark that encompasses ten state-of-the-art algorithms, a diverse array of distribution shifts, and two evaluation protocols.
  arXiv  Detail & Related papers  (2023-06-06T09:35:29Z)
• DELTA: degradation-free fully test-time adaptation [59.74287982885375]
  We find that two unfavorable defects are concealed in the prevalent adaptation methodologies like test-time batch normalization (BN) and self-learning. First, we reveal that the normalization statistics in test-time BN are completely affected by the currently received test samples, resulting in inaccurate estimates. Second, we show that during test-time adaptation, the parameter update is biased towards some dominant classes.
  arXiv  Detail & Related papers  (2023-01-30T15:54:00Z)
• A Probabilistic Framework for Lifelong Test-Time Adaptation [34.07074915005366]
  Test-time adaptation (TTA) is the problem of updating a pre-trained source model at inference time given test input(s) from a different target domain. We present PETAL (Probabilistic lifElong Test-time Adaptation with seLf-training prior), which solves lifelong TTA using a probabilistic approach. Our method achieves better results than the current state-of-the-art for online lifelong test-time adaptation across various benchmarks.
  arXiv  Detail & Related papers  (2022-12-19T18:42:19Z)
• CAFA: Class-Aware Feature Alignment for Test-Time Adaptation [50.26963784271912]
  Test-time adaptation (TTA) aims to address this challenge by adapting a model to unlabeled data at test time. We propose a simple yet effective feature alignment loss, termed as Class-Aware Feature Alignment (CAFA), which simultaneously encourages a model to learn target representations in a class-discriminative manner.
  arXiv  Detail & Related papers  (2022-06-01T03:02:07Z)
• Efficient Test-Time Model Adaptation without Forgetting [60.36499845014649]
  Test-time adaptation seeks to tackle potential distribution shifts between training and testing data. We propose an active sample selection criterion to identify reliable and non-redundant samples. We also introduce a Fisher regularizer to constrain important model parameters from drastic changes.
  arXiv  Detail & Related papers  (2022-04-06T06:39:40Z)
• On-the-Fly Test-time Adaptation for Medical Image Segmentation [63.476899335138164]
  Adapting the source model to target data distribution at test-time is an efficient solution for the data-shift problem. We propose a new framework called Adaptive UNet where each convolutional block is equipped with an adaptive batch normalization layer. During test-time, the model takes in just the new test image and generates a domain code to adapt the features of source model according to the test data.
  arXiv  Detail & Related papers  (2022-03-10T18:51:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.