AttenDence: Maximizing Attention Confidence for Test Time Adaptation

• URL: http://arxiv.org/abs/2511.18925v1
• Date: Mon, 24 Nov 2025 09:32:01 GMT
• Title: AttenDence: Maximizing Attention Confidence for Test Time Adaptation
• Abstract summary: Test-time adaptation (TTA) enables models to adapt to distribution shifts at inference time.<n>We propose minimizing the entropy of attention distributions from the CLS token to image patches as a novel TTA objective.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Test-time adaptation (TTA) enables models to adapt to distribution shifts at inference time. While entropy minimization over the output distribution has proven effective for TTA, transformers offer an additional unsupervised learning signal through their attention mechanisms. We propose minimizing the entropy of attention distributions from the CLS token to image patches as a novel TTA objective.This approach encourages the model to attend more confidently to relevant image regions under distribution shift and is effective even when only a single test image is available. We demonstrate that attention entropy minimization improves robustness across diverse corruption types while not hurting performance on clean data on a single sample stream of images at test time.

Related papers

• Test-Time Curriculum for Open-Set AIGC Detection [31.702537861903014]
  Test-Time Curriculum (TTC) adapts a detector on unlabeled test data through curriculum-based self-training.<n>TTC starts from highly reliable pseudo-labeled samples and incorporates harder yet informative cases.<n>Cross-Scale Pseudo-Label Refinement aggregates complementary evidence across multiple resolutions for more reliable adaptation.
  arXiv  Detail & Related papers  (2026-08-01T09:42:45Z)
• Mitigating the ID-OOD Tradeoff in Open-Set Test-Time Adaptation [52.620992698137236]
  Open-set test-time adaptation (OSTTA) addresses the challenge of adapting models to new environments.<n>In this paper, we first analyze the limitations of entropy in OSTTA and then introduce an angular loss to regulate feature norm magnitudes.<n>Our method achieves strong OOD detection while maintaining high ID classification performance.
  arXiv  Detail & Related papers  (2026-04-02T04:00:57Z)
• ProtoDCS: Towards Robust and Efficient Open-Set Test-Time Adaptation for Vision-Language Models [32.840734752367275]
  Prototype-based Double-Check Separation (ProtoDCS) is a robust framework for OSTTA.<n>It separates csID and csOOD samples, enabling safe and efficient adaptation of Vision-Language Models to csID data.<n>ProtoDCS significantly boosts both known-class accuracy and OOD detection metrics.
  arXiv  Detail & Related papers  (2026-02-27T03:39:02Z)
• CLIPTTA: Robust Contrastive Vision-Language Test-Time Adaptation [15.732351927470452]
  Vision-language models (VLMs) like CLIP exhibit strong zero-shot capabilities but often fail to generalize under distribution shifts.<n>Test-time adaptation (TTA) allows models to update at inference time without labeled data, typically via entropy minimization.<n>We propose CLIPTTA, a new gradient-based TTA method for vision-language models that leverages a soft contrastive loss aligned with CLIP's pre-training objective.
  arXiv  Detail & Related papers  (2025-07-18T18:32:17Z)
• Uniformity First: Uniformity-aware Test-time Adaptation of Vision-language Models against Image Corruption [4.792851066169872]
  We find that vision-language models still suffer when they face datasets with large gaps from training ones.<n>We propose a novel method called information-balanced TTA (UnInfo) to make models robust to sensor degradation.
  arXiv  Detail & Related papers  (2025-05-19T09:47:46Z)
• Enhancing Test Time Adaptation with Few-shot Guidance [62.49199492255226]
  Deep neural networks often encounter significant performance drops while facing with domain shifts between training (source) and test (target) data.<n>Test Time Adaptation (TTA) methods have been proposed to adapt pre-trained source model to handle out-of-distribution streaming target data.<n>We develop Few-Shot Test Time Adaptation (FS-TTA), a novel and practical setting that utilizes a few-shot support set on top of TTA.
  arXiv  Detail & Related papers  (2024-09-02T15:50:48Z)
• Adaptive Bidirectional Displacement for Semi-Supervised Medical Image Segmentation [11.195959019678314]
  Consistency learning is a central strategy to tackle unlabeled data in semi-supervised medical image segmentation. In this paper, we propose an Adaptive Bidirectional Displacement approach to solve the above challenge.
  arXiv  Detail & Related papers  (2024-05-01T08:17:43Z)
• Test-Time Model Adaptation with Only Forward Passes [68.11784295706995]
  Test-time adaptation has proven effective in adapting a given trained model to unseen test samples with potential distribution shifts. We propose a test-time Forward-Optimization Adaptation (FOA) method. FOA runs on quantized 8-bit ViT, outperforms gradient-based TENT on full-precision 32-bit ViT, and achieves an up to 24-fold memory reduction on ImageNet-C.
  arXiv  Detail & Related papers  (2024-04-02T05:34:33Z)
• Each Test Image Deserves A Specific Prompt: Continual Test-Time Adaptation for 2D Medical Image Segmentation [14.71883381837561]
  Cross-domain distribution shift is a significant obstacle to deploying the pre-trained semantic segmentation model in real-world applications. Test-time adaptation has proven its effectiveness in tackling the cross-domain distribution shift during inference. We propose the Visual Prompt-based Test-Time Adaptation (VPTTA) method to train a specific prompt for each test image to align the statistics in the batch normalization layers.
  arXiv  Detail & Related papers  (2023-11-30T09:03:47Z)
• Learning to Mask and Permute Visual Tokens for Vision Transformer Pre-Training [55.12082817901671]
  We propose a new self-supervised pre-training approach, named Masked and Permuted Vision Transformer (MaPeT)<n>MaPeT employs autoregressive and permuted predictions to capture intra-patch dependencies.<n>Our results demonstrate that MaPeT achieves competitive performance on ImageNet, compared to baselines and competitors under the same model setting.
  arXiv  Detail & Related papers  (2023-06-12T18:12:19Z)
• Masked Images Are Counterfactual Samples for Robust Fine-tuning [77.82348472169335]
  Fine-tuning deep learning models can lead to a trade-off between in-distribution (ID) performance and out-of-distribution (OOD) robustness. We propose a novel fine-tuning method, which uses masked images as counterfactual samples that help improve the robustness of the fine-tuning model.
  arXiv  Detail & Related papers  (2023-03-06T11:51:28Z)
• Efficient Test-Time Model Adaptation without Forgetting [60.36499845014649]
  Test-time adaptation seeks to tackle potential distribution shifts between training and testing data. We propose an active sample selection criterion to identify reliable and non-redundant samples. We also introduce a Fisher regularizer to constrain important model parameters from drastic changes.
  arXiv  Detail & Related papers  (2022-04-06T06:39:40Z)
• Test-time Adaptation with Slot-Centric Models [63.981055778098444]
  Slot-TTA is a semi-supervised scene decomposition model that at test time is adapted per scene through gradient descent on reconstruction or cross-view synthesis objectives. We show substantial out-of-distribution performance improvements against state-of-the-art supervised feed-forward detectors, and alternative test-time adaptation methods.
  arXiv  Detail & Related papers  (2022-03-21T17:59:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.