ProtoTTA: Prototype-Guided Test-Time Adaptation

• URL: http://arxiv.org/abs/2604.15494v1
• Date: Thu, 16 Apr 2026 19:52:57 GMT
• Title: ProtoTTA: Prototype-Guided Test-Time Adaptation
• Abstract summary: We introduce ProtoTTA, a framework for prototypical models that leverage intermediate prototype signals rather than relying solely on model outputs.<n>ProtoTTA minimizes the entropy of the prototype-similarity distribution to encourage more confident and prototype-specific activations on shifted data.<n>We also introduce novel interpretability metrics and a vision-language minimization model (VLM) evaluation framework to explain TTA dynamics.
• Score: 8.178991486617972
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep networks that rely on prototypes-interpretable representations that can be related to the model input-have gained significant attention for balancing high accuracy with inherent interpretability, which makes them suitable for critical domains such as healthcare. However, these models are limited by their reliance on training data, which hampers their robustness to distribution shifts. While test-time adaptation (TTA) improves the robustness of deep networks by updating parameters and statistics, the prototypes of interpretable models have not been explored for this purpose. We introduce ProtoTTA, a general framework for prototypical models that leverages intermediate prototype signals rather than relying solely on model outputs. ProtoTTA minimizes the entropy of the prototype-similarity distribution to encourage more confident and prototype-specific activations on shifted data. To maintain stability, we employ geometric filtering to restrict updates to samples with reliable prototype activations, regularized by prototype-importance weights and model-confidence scores. Experiments across four prototypical backbones on four diverse benchmarks spanning fine-grained vision, histopathology, and NLP demonstrate that ProtoTTA improves robustness over standard output entropy minimization while restoring correct semantic focus in prototype activations. We also introduce novel interpretability metrics and a vision-language model (VLM) evaluation framework to explain TTA dynamics, confirming ProtoTTA restores human-aligned semantic focus and correlates reliably with VLM-rated reasoning quality. Code is available at: https://github.com/DeepRCL/ProtoTTA.

Related papers

• Distilling Image Prototypes for Guided Test-Time Adaptation [24.945006924099086]
  Test-Time Adaptation (TTA) enhances the robustness of models against distribution shifts.<n>It faces two critical challenges: error accumulation from noisy pseudo-labels and catastrophic forgetting of source knowledge.<n>This paper proposes a novel framework, Distilling Image Prototype for Guided Test-Time Adaptation (DIPTTA)
  arXiv  Detail & Related papers  (2026-09-09T05:27:09Z)
• Reliable Neural Collapse Approximation for Open-World Test-Time Adaptation [95.55552104238787]
  Test-Time Adaptation (TTA) methods aim to bridge the domain gap between the source and target domains.<n>Traditional TTA methods become ineffective when the label distribution shift occurs.<n>We introduce a new method named Reliable Neural Collapse approximation (ReNC) for Open-World Test-Time Adaptation.
  arXiv  Detail & Related papers  (2026-08-20T10:57:44Z)
• Prototype Transformer: Towards Language Model Architectures Interpretable by Design [37.30649990861446]
  We introduce the Prototype Transformer (ProtoT) -- an autoregressive LM architecture based on prototypes.<n>ProtoT works by means of two-way communication between the input sequence and the prototypes.<n>It provides the potential to interpret the model's reasoning and allow for targeted edits of its behavior.
  arXiv  Detail & Related papers  (2026-02-12T11:43:39Z)
• Proto-Former: Unified Facial Landmark Detection by Prototype Transformer [77.47431726595111]
  Proto-Former is a unified, adaptive, end-to-end facial landmark detection framework.<n>It enables joint training across multiple datasets within a unified architecture.<n>Proto-Former achieves superior performance compared to existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-10-17T06:00:25Z)
• Model Correlation Detection via Random Selection Probing [62.093777777813756]
  Existing similarity-based methods require access to model parameters or produce scores without thresholds.<n>We introduce Random Selection Probing (RSP), a hypothesis-testing framework that formulates model correlation detection as a statistical test.<n>RSP produces rigorous p-values that quantify evidence of correlation.
  arXiv  Detail & Related papers  (2025-09-29T01:40:26Z)
• ProtoTS: Learning Hierarchical Prototypes for Explainable Time Series Forecasting [25.219624871510376]
  We propose ProtoTS, a novel interpretable forecasting framework that achieves both high accuracy and transparent decision-making.<n>ProtoTS computes instance-prototype similarity based on a denoised representation that preserves abundant heterogeneous information.<n> Experiments on multiple realistic benchmarks, including a newly released LOF dataset, show that ProtoTS not only exceeds existing methods in forecast accuracy but also delivers expert-steerable interpretations.
  arXiv  Detail & Related papers  (2025-09-27T07:10:21Z)
• Probabilistic Prototype Calibration of Vision-Language Models for Generalized Few-shot Semantic Segmentation [75.18058114915327]
  Generalized Few-Shot Semanticnative (GFSS) aims to extend a segmentation model to novel classes with only a few annotated examples.<n>We propose FewCLIP, a probabilistic prototype calibration framework over multi-modal prototypes from the pretrained CLIP.<n>We show FewCLIP significantly outperforms state-of-the-art approaches across both GFSS and class-incremental setting.
  arXiv  Detail & Related papers  (2025-06-28T18:36:22Z)
• Bidirectional Prototype-Reward co-Evolution for Test-Time Adaptation of Vision-Language Models [38.63571023556356]
  Test-time adaptation (TTA) is crucial in maintaining performance of Vision Language Models (VLMs) when facing distribution shifts.<n>We propose underlineBidirectional Prototype-Reward co-Evolution (BPRE)<n>BPRE integrates feature quality assessment with prototype evolution via a synergistic feedback loop.<n>Our model consistently achieves superior performance compared to other SOTA methods, and advances VLM generalization capabilities.
  arXiv  Detail & Related papers  (2025-03-12T13:40:33Z)
• Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction [88.65168366064061]
  We introduce Discrete Denoising Posterior Prediction (DDPP), a novel framework that casts the task of steering pre-trained MDMs as a problem of probabilistic inference. Our framework leads to a family of three novel objectives that are all simulation-free, and thus scalable. We substantiate our designs via wet-lab validation, where we observe transient expression of reward-optimized protein sequences.
  arXiv  Detail & Related papers  (2024-10-10T17:18:30Z)
• Test-Time Model Adaptation with Only Forward Passes [68.11784295706995]
  Test-time adaptation has proven effective in adapting a given trained model to unseen test samples with potential distribution shifts. We propose a test-time Forward-Optimization Adaptation (FOA) method. FOA runs on quantized 8-bit ViT, outperforms gradient-based TENT on full-precision 32-bit ViT, and achieves an up to 24-fold memory reduction on ImageNet-C.
  arXiv  Detail & Related papers  (2024-04-02T05:34:33Z)
• Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Models [19.683461002518147]
  Test-Time Prototype Shifting (TPS) is a pioneering approach designed to adapt vision-language models to test datasets using unlabeled test inputs.<n>TPS not only facilitates optimization-free prototype reuse for subsequent predictions but also enables seamless integration with current advancements in prompt engineering.<n>A notable aspect of our framework is its significantly reduced memory and computational demands when compared to conventional text-prompt tuning methods.
  arXiv  Detail & Related papers  (2024-03-19T17:54:34Z)
• Prototypical Contrastive Learning of Unsupervised Representations [171.3046900127166]
  Prototypical Contrastive Learning (PCL) is an unsupervised representation learning method. PCL implicitly encodes semantic structures of the data into the learned embedding space. PCL outperforms state-of-the-art instance-wise contrastive learning methods on multiple benchmarks.
  arXiv  Detail & Related papers  (2020-05-11T09:53:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.