Uncover and Unlearn Nuisances: Agnostic Fully Test-Time Adaptation

• URL: http://arxiv.org/abs/2511.12491v1
• Date: Sun, 16 Nov 2025 08:01:50 GMT
• Title: Uncover and Unlearn Nuisances: Agnostic Fully Test-Time Adaptation
• Authors: Ponhvoan Srey, Yaxin Shi, Hangwei Qian, Jing Li, Ivor W. Tsang,
• Abstract summary: Fully Test-Time Adaptation (FTTA) addresses domain shifts without access to source data and training protocols of the pre-trained models.<n>Traditional strategies that align source and target feature distributions are infeasible in FTTA due to the absence of training data and unpredictable target domains.<n>We propose Agnostic FTTA as a novel formulation that enables the usage of off-the-shelf domain transformations during test-time to enable direct generalization to unforeseeable target data.
• Score: 46.46865123564661
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fully Test-Time Adaptation (FTTA) addresses domain shifts without access to source data and training protocols of the pre-trained models. Traditional strategies that align source and target feature distributions are infeasible in FTTA due to the absence of training data and unpredictable target domains. In this work, we exploit a dual perspective on FTTA, and propose Agnostic FTTA (AFTTA) as a novel formulation that enables the usage of off-the-shelf domain transformations during test-time to enable direct generalization to unforeseeable target data. To address this, we develop an uncover-and-unlearn approach. First, we uncover potential unwanted shifts between source and target domains by simulating them through predefined mappings and consider them as nuisances. Then, during test-time prediction, the model is enforced to unlearn these nuisances by regularizing the consequent shifts in latent representations and label predictions. Specifically, a mutual information-based criterion is devised and applied to guide nuisances unlearning in the feature space and encourage confident and consistent prediction in label space. Our proposed approach explicitly addresses agnostic domain shifts, enabling superior model generalization under FTTA constraints. Extensive experiments on various tasks, involving corruption and style shifts, demonstrate that our method consistently outperforms existing approaches.

Related papers

• Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignment [16.352863226512984]
  Test-time adaptation (TTA) enhances the zero-shot robustness under distribution shifts by leveraging unlabeled test data during inference.<n>Most methods rely on backpropagation or iterative optimization, which limits scalability and hinders real-time deployment.<n>We propose ADAPT, an Advanced Distribution-Aware and back propagation-free Test-time adaptation method.
  arXiv  Detail & Related papers  (2025-08-21T13:42:49Z)
• Proposal-Level Unsupervised Domain Adaptation for Open World Unbiased Detector [35.334125159092025]
  We build an unbiased foreground predictor by re-formulating the task under Unsupervised Domain Adaptation. We adopt the simple and effective self-training method to learn a predictor based on the domain-invariant foreground features. Our approach's pipeline can adapt to various detection frameworks and UDA methods, empirically validated by OWOD evaluation.
  arXiv  Detail & Related papers  (2023-11-04T07:46:45Z)
• Uncertainty-guided Source-free Domain Adaptation [77.3844160723014]
  Source-free domain adaptation (SFDA) aims to adapt a classifier to an unlabelled target data set by only using a pre-trained source model. We propose quantifying the uncertainty in the source model predictions and utilizing it to guide the target adaptation.
  arXiv  Detail & Related papers  (2022-08-16T08:03:30Z)
• CAFA: Class-Aware Feature Alignment for Test-Time Adaptation [50.26963784271912]
  Test-time adaptation (TTA) aims to address this challenge by adapting a model to unlabeled data at test time. We propose a simple yet effective feature alignment loss, termed as Class-Aware Feature Alignment (CAFA), which simultaneously encourages a model to learn target representations in a class-discriminative manner.
  arXiv  Detail & Related papers  (2022-06-01T03:02:07Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• A Curriculum-style Self-training Approach for Source-Free Semantic Segmentation [91.13472029666312]
  We propose a curriculum-style self-training approach for source-free domain adaptive semantic segmentation. Our method yields state-of-the-art performance on source-free semantic segmentation tasks for both synthetic-to-real and adverse conditions.
  arXiv  Detail & Related papers  (2021-06-22T10:21:39Z)
• Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer [137.36099660616975]
  Unsupervised adaptation adaptation (UDA) aims to transfer knowledge from a related but different well-labeled source domain to a new unlabeled target domain. Most existing UDA methods require access to the source data, and thus are not applicable when the data are confidential and not shareable due to privacy concerns. This paper aims to tackle a realistic setting with only a classification model available trained over, instead of accessing to the source data.
  arXiv  Detail & Related papers  (2020-12-14T07:28:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.