Efficient Test-Time Model Adaptation without Forgetting
- URL: http://arxiv.org/abs/2204.02610v1
- Date: Wed, 6 Apr 2022 06:39:40 GMT
- Title: Efficient Test-Time Model Adaptation without Forgetting
- Authors: Shuaicheng Niu and Jiaxiang Wu and Yifan Zhang and Yaofo Chen and
Shijian Zheng and Peilin Zhao and Mingkui Tan
- Abstract summary: Test-time adaptation seeks to tackle potential distribution shifts between training and testing data.
We propose an active sample selection criterion to identify reliable and non-redundant samples.
We also introduce a Fisher regularizer to constrain important model parameters from drastic changes.
- Score: 60.36499845014649
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Test-time adaptation (TTA) seeks to tackle potential distribution shifts
between training and testing data by adapting a given model w.r.t. any testing
sample. This task is particularly important for deep models when the test
environment changes frequently. Although some recent attempts have been made to
handle this task, we still face two practical challenges: 1) existing methods
have to perform backward computation for each test sample, resulting in
unbearable prediction cost to many applications; 2) while existing TTA
solutions can significantly improve the test performance on out-of-distribution
data, they often suffer from severe performance degradation on in-distribution
data after TTA (known as catastrophic forgetting). In this paper, we point out
that not all the test samples contribute equally to model adaptation, and
high-entropy ones may lead to noisy gradients that could disrupt the model.
Motivated by this, we propose an active sample selection criterion to identify
reliable and non-redundant samples, on which the model is updated to minimize
the entropy loss for test-time adaptation. Furthermore, to alleviate the
forgetting issue, we introduce a Fisher regularizer to constrain important
model parameters from drastic changes, where the Fisher importance is estimated
from test samples with generated pseudo labels. Extensive experiments on
CIFAR-10-C, ImageNet-C, and ImageNet-R verify the effectiveness of our proposed
method.
Related papers
- DOTA: Distributional Test-Time Adaptation of Vision-Language Models [52.98590762456236]
Training-free test-time dynamic adapter (TDA) is a promising approach to address this issue.
We propose a simple yet effective method for DistributiOnal Test-time Adaptation (Dota)
Dota continually estimates the distributions of test samples, allowing the model to continually adapt to the deployment environment.
arXiv Detail & Related papers (2024-09-28T15:03:28Z) - Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting [55.17761802332469]
Test-time adaptation (TTA) seeks to tackle potential distribution shifts between training and test data by adapting a given model w.r.t. any test sample.
Prior methods perform backpropagation for each test sample, resulting in unbearable optimization costs to many applications.
We propose an Efficient Anti-Forgetting Test-Time Adaptation (EATA) method which develops an active sample selection criterion to identify reliable and non-redundant samples.
arXiv Detail & Related papers (2024-03-18T05:49:45Z) - Improving Entropy-Based Test-Time Adaptation from a Clustering View [15.157208389691238]
We introduce a new clustering perspective on the entropy-based TTA.
We propose to improve EBTTA from the assignment step and the updating step, where robust label assignment, similarity-preserving constraint, sample selection, and gradient accumulation are proposed.
Experimental results demonstrate that our method can achieve consistent improvements on various datasets.
arXiv Detail & Related papers (2023-10-31T10:10:48Z) - REALM: Robust Entropy Adaptive Loss Minimization for Improved
Single-Sample Test-Time Adaptation [5.749155230209001]
Fully-test-time adaptation (F-TTA) can mitigate performance loss due to distribution shifts between train and test data.
We present a general framework for improving robustness of F-TTA to noisy samples, inspired by self-paced learning and robust loss functions.
arXiv Detail & Related papers (2023-09-07T18:44:58Z) - DELTA: degradation-free fully test-time adaptation [59.74287982885375]
We find that two unfavorable defects are concealed in the prevalent adaptation methodologies like test-time batch normalization (BN) and self-learning.
First, we reveal that the normalization statistics in test-time BN are completely affected by the currently received test samples, resulting in inaccurate estimates.
Second, we show that during test-time adaptation, the parameter update is biased towards some dominant classes.
arXiv Detail & Related papers (2023-01-30T15:54:00Z) - TTAPS: Test-Time Adaption by Aligning Prototypes using Self-Supervision [70.05605071885914]
We propose a novel modification of the self-supervised training algorithm SwAV that adds the ability to adapt to single test samples.
We show the success of our method on the common benchmark dataset CIFAR10-C.
arXiv Detail & Related papers (2022-05-18T05:43:06Z) - MEMO: Test Time Robustness via Adaptation and Augmentation [131.28104376280197]
We study the problem of test time robustification, i.e., using the test input to improve model robustness.
Recent prior works have proposed methods for test time adaptation, however, they each introduce additional assumptions.
We propose a simple approach that can be used in any test setting where the model is probabilistic and adaptable.
arXiv Detail & Related papers (2021-10-18T17:55:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.