Dual Distribution Estimation for Zero-shot Noisy Test-Time Adaptation with VLMs

• URL: http://arxiv.org/abs/2606.25758v1
• Date: Wed, 24 Jun 2026 12:32:01 GMT
• Title: Dual Distribution Estimation for Zero-shot Noisy Test-Time Adaptation with VLMs
• Abstract summary: We propose a novel framework named Dual Distribution Estimation (DDE)<n>DDE incorporates two novel modules: Positive Feature Distribution Estimation (PFDE) and Negative Label Distribution Estimation (NLDE)<n>On the large-scale ImageNet benchmark, DDE achieves an improvement of 3.70% in harmonic mean accuracy and reduces the FPR95 for OOD detection by 6.20%.
• Score: 26.58533514461074
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While test-time adaptation (TTA) empowers vision-language models to adapt without costly retraining, it remains highly vulnerable to out-of-distribution (OOD) outliers prevalent in real-world applications. This discrepancy motivates Noisy TTA (NTTA), an online task to filter noisy OOD samples on the fly while maximizing in-distribution (ID) classification accuracy. Existing zero-shot NTTA approaches typically rely on test-time discriminative training, leading to overconfident misclassifications and significantly degraded inference efficiency. To address these limitations, we propose a novel framework named Dual Distribution Estimation (DDE), shifting the zero-shot NTTA paradigm from instance-level learning to training-free Gaussian distribution modeling. DDE incorporates two novel modules: Positive Feature Distribution Estimation (PFDE) and Negative Label Distribution Estimation (NLDE). PFDE explicitly models class-wise inclusion and exclusion Gaussian distributions to formulate a calibrated contrastive score, robustly enhancing ID accuracy. In parallel, NLDE improves OOD identification by explicitly modeling the negative label distribution to mine highly discriminative labels, effectively mitigating spurious correlations. Extensive experiments show that on the large-scale ImageNet benchmark, DDE achieves an improvement of 3.70\% in harmonic mean accuracy and reduces the FPR95 for OOD detection by 6.20\%, while ensuring highly scalable and efficient online inference. Furthermore, DDE is zero-shot and training-free, demonstrating remarkable robustness in data-scarce scenarios. Codes are available at https://github.com/ZhuWenjie98/DDE.

Related papers

• MixSD: Mixed Contextual Self-Distillation for Knowledge Injection [29.7616760417696]
  Supervised fine-tuning (SFT) is widely used to inject new knowledge into language models.<n>We argue this forgetting arises because fine-tuning targets from humans or external systems diverge from the model's autoregressive distribution.<n>We propose MixSD, a simple external-teacher-free method for distribution-aligned knowledge injection.
  arXiv  Detail & Related papers  (2026-05-16T07:57:09Z)
• Mitigating the ID-OOD Tradeoff in Open-Set Test-Time Adaptation [52.620992698137236]
  Open-set test-time adaptation (OSTTA) addresses the challenge of adapting models to new environments.<n>In this paper, we first analyze the limitations of entropy in OSTTA and then introduce an angular loss to regulate feature norm magnitudes.<n>Our method achieves strong OOD detection while maintaining high ID classification performance.
  arXiv  Detail & Related papers  (2026-04-02T04:00:57Z)
• ProtoDCS: Towards Robust and Efficient Open-Set Test-Time Adaptation for Vision-Language Models [32.840734752367275]
  Prototype-based Double-Check Separation (ProtoDCS) is a robust framework for OSTTA.<n>It separates csID and csOOD samples, enabling safe and efficient adaptation of Vision-Language Models to csID data.<n>ProtoDCS significantly boosts both known-class accuracy and OOD detection metrics.
  arXiv  Detail & Related papers  (2026-02-27T03:39:02Z)
• Cross-modal Proxy Evolving for OOD Detection with Vision-Language Models [59.242742594156546]
  CoEvo is a test-time framework that performs bidirectional, sample-conditioned adaptation of both textual and visual proxies.<n>CoEvo achieves state-of-the-art performance, improving AUROC by 1.33% and reducing FPR95 by 45.98% on ImageNet-1K compared to strong negative-label baselines.
  arXiv  Detail & Related papers  (2026-01-13T12:08:26Z)
• Noisy Test-Time Adaptation in Vision-Language Models [73.14136220844156]
  Test-time adaptation (TTA) aims to address distribution shifts between source and target data by relying solely on target data during testing.<n>This paper introduces Zero-Shot Noisy TTA (ZS-NTTA), focusing on adapting the model to target data with noisy samples during test-time in a zero-shot manner.<n>We introduce the Adaptive Noise Detector (AdaND), which utilizes the frozen model's outputs as pseudo-labels to train a noise detector.
  arXiv  Detail & Related papers  (2025-02-20T14:37:53Z)
• DisCoPatch: Taming Adversarially-driven Batch Statistics for Improved Out-of-Distribution Detection [11.332987462182713]
  In adversarial discriminators trained with Batch Normalization (BN), real and adversarial samples form distinct domains with unique batch statistics.<n>We introduce DisCoPatch, an unsupervised Adversarial Variational Autoencoder framework that harnesses this mechanism.<n>DisCoPatch achieves state-of-the-art results in public OOD detection benchmarks.
  arXiv  Detail & Related papers  (2025-01-14T10:49:26Z)
• Robust Fine-tuning of Zero-shot Models via Variance Reduction [56.360865951192324]
  When fine-tuning zero-shot models, our desideratum is for the fine-tuned model to excel in both in-distribution (ID) and out-of-distribution (OOD) We propose a sample-wise ensembling technique that can simultaneously attain the best ID and OOD accuracy without the trade-offs.
  arXiv  Detail & Related papers  (2024-11-11T13:13:39Z)
• Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution [67.9215891673174]
  We propose score entropy as a novel loss that naturally extends score matching to discrete spaces. We test our Score Entropy Discrete Diffusion models on standard language modeling tasks.
  arXiv  Detail & Related papers  (2023-10-25T17:59:12Z)
• Diffusion Denoising Process for Perceptron Bias in Out-of-distribution Detection [67.49587673594276]
  We introduce a new perceptron bias assumption that suggests discriminator models are more sensitive to certain features of the input, leading to the overconfidence problem. We demonstrate that the diffusion denoising process (DDP) of DMs serves as a novel form of asymmetric, which is well-suited to enhance the input and mitigate the overconfidence problem. Our experiments on CIFAR10, CIFAR100, and ImageNet show that our method outperforms SOTA approaches.
  arXiv  Detail & Related papers  (2022-11-21T08:45:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.