Test-Time Logit Prompting for Source-Free Missing Modality Adaptation

• URL: http://arxiv.org/abs/2609.02039v1
• Date: Wed, 02 Sep 2026 03:13:06 GMT
• Title: Test-Time Logit Prompting for Source-Free Missing Modality Adaptation
• Abstract summary: We propose Test-Time Logit Prompting (TLP) as a lightweight source-free test-time adaptation framework for visual recognition with missing modalities.<n>TLP consistently enhances recognition performance under missing-modality scenarios, achieving up to 8% improvements.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language models (VLMs) have achieved remarkable performance by leveraging complementary information from large-scale image-text pairs. However, missing-modality inputs are commonly encountered during real-world deployment, often leading to significant performance degradation. Existing methods primarily enhance model robustness by learning modality compensation strategies from source training data. However, their reliance on source training data makes them difficult to apply when original data are unavailable due to privacy, storage, or accessibility constraints, such as clinical applications and personalized AI services. This raises an important yet underexplored question: can VLMs be efficiently adapted at test time for visual recognition with missing modalities without accessing source training data? To this end, we propose Test-Time Logit Prompting (TLP), a lightweight source-free test-time adaptation framework for visual recognition with missing modalities. To address missing-induced prediction shifts, TLP optimizes logit prompts with uncertainty-aware adjustment and modality-complete consistency regularization, adaptively adjusting prediction confidence while preserving semantic consistency. Extensive experiments across diverse vision-language benchmarks demonstrate that TLP consistently enhances recognition performance under missing-modality scenarios, achieving up to 8\% improvements while requiring only hundreds of tunable parameters and a few test-time optimization steps.

Related papers

• Learning from Reliable Latent Prompts for Visual Recognition with Missing Modalities [0.0]
  Large-scale multimodal models (LMMs) have achieved superior performance in visual recognition by synergizing information across diverse, massive-scale paired modalities.<n>In real-world scenarios, however, missing-modality inputs are ubiquitous, causing models optimized for modality-complete data to exhibit precipitous performance degradation.<n>Existing research has introduced prompt learning to mitigate this issue, typically by generating dynamic prompts from instance-level features.<n>As higher missing rates increase the proportion of incomplete modalities, the resulting instability in prompt learning limits the model's performance.
  arXiv  Detail & Related papers  (2026-06-29T17:36:47Z)
• Pretrain-then-Adapt: Uncertainty-Aware Test-Time Adaptation for Text-based Person Search [26.52140212357349]
  Text-based person search faces inherent limitations due to data scarcity, driven by stringent privacy constraints and the high cost of manual annotation.<n>Existing methods usually rely on a Pretrain-then-Finetune paradigm, where models are first pretrained on synthetic person-caption data.<n>We propose a new Pretrain-then-Adapt paradigm that eliminates reliance on extensive target-domain supervision through an offline test-time adaptation manner.
  arXiv  Detail & Related papers  (2026-04-07T07:48:15Z)
• Doubly Debiased Test-Time Prompt Tuning for Vision-Language Models [43.35073848153914]
  Test-time prompt tuning for vision-language models has demonstrated impressive generalization capabilities under zero-shot settings.<n>We analyze the underlying causes of prompt optimization bias from both the model and data perspectives.
  arXiv  Detail & Related papers  (2025-11-12T09:35:31Z)
• Advancing Reliable Test-Time Adaptation of Vision-Language Models under Visual Variations [67.35596444651037]
  Vision-language models (VLMs) exhibit remarkable zero-shot capabilities but struggle with distribution shifts in downstream tasks when labeled data is unavailable.<n>We propose a Reliable Test-time Adaptation (ReTA) method that enhances reliability from two perspectives.
  arXiv  Detail & Related papers  (2025-07-13T05:37:33Z)
• In-context Demonstration Matters: On Prompt Optimization for Pseudo-Supervision Refinement [71.60563181678323]
  Large language models (LLMs) have achieved great success across diverse tasks, and fine-tuning is sometimes needed to further enhance generation quality.<n>To handle these challenges, a direct solution is to generate high-confidence'' data from unsupervised downstream tasks.<n>We propose a novel approach, pseudo-supervised demonstrations aligned prompt optimization (PAPO) algorithm, which jointly refines both the prompt and the overall pseudo-supervision.
  arXiv  Detail & Related papers  (2024-10-04T03:39:28Z)
• Data Adaptive Traceback for Vision-Language Foundation Models in Image Classification [34.37262622415682]
  We propose a new adaptation framework called Data Adaptive Traceback. Specifically, we utilize a zero-shot-based method to extract the most downstream task-related subset of the pre-training data. We adopt a pseudo-label-based semi-supervised technique to reuse the pre-training images and a vision-language contrastive learning method to address the confirmation bias issue in semi-supervised learning.
  arXiv  Detail & Related papers  (2024-07-11T18:01:58Z)
• Test-Time Adaptation for Combating Missing Modalities in Egocentric Videos [92.38662956154256]
  Real-world applications often face challenges with incomplete modalities due to privacy concerns, efficiency needs, or hardware issues.<n>We propose a novel approach to address this issue at test time without requiring retraining.<n>MiDl represents the first self-supervised, online solution for handling missing modalities exclusively at test time.
  arXiv  Detail & Related papers  (2024-04-23T16:01:33Z)
• Cluster-level pseudo-labelling for source-free cross-domain facial expression recognition [94.56304526014875]
  We propose the first Source-Free Unsupervised Domain Adaptation (SFUDA) method for Facial Expression Recognition (FER) Our method exploits self-supervised pretraining to learn good feature representations from the target data. We validate the effectiveness of our method in four adaptation setups, proving that it consistently outperforms existing SFUDA methods when applied to FER.
  arXiv  Detail & Related papers  (2022-10-11T08:24:50Z)
• CAFA: Class-Aware Feature Alignment for Test-Time Adaptation [50.26963784271912]
  Test-time adaptation (TTA) aims to address this challenge by adapting a model to unlabeled data at test time. We propose a simple yet effective feature alignment loss, termed as Class-Aware Feature Alignment (CAFA), which simultaneously encourages a model to learn target representations in a class-discriminative manner.
  arXiv  Detail & Related papers  (2022-06-01T03:02:07Z)
• Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognition [65.84978547406753]
  Test-time Adaptation aims to adapt the model trained on source domains to yield better predictions for test samples. Single-Utterance Test-time Adaptation (SUTA) is the first TTA study in speech area to our best knowledge.
  arXiv  Detail & Related papers  (2022-03-27T06:38:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.