Towards Purified Multi-Label Test-Time Adaptation of Vision-Language Models

• URL: http://arxiv.org/abs/2608.25653v1
• Date: Wed, 26 Aug 2026 11:36:11 GMT
• Title: Towards Purified Multi-Label Test-Time Adaptation of Vision-Language Models
• Abstract summary: PuRF is a novel PuRiFication-driven cache-based method for multi-label test-time adaptation of vision-language models.<n>We show PuRF consistently outperforms state-of-the-art methods, achieving a notable 4.05% mAP improvement on ViT-B/32 across five datasets.
• Score: 81.26524429852992
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Test-time adaptation (TTA) has been widely explored in single-label recognition, effectively mitigating distribution shifts, especially when combined with vision-language models. However, real-world images often contain multiple objects, while the more practical multi-label test-time adaptation (MLTTA) has received little attention so far. Recent cache-based TTA methods have shown promising efficiency and effectiveness, yet directly extending them to multi-label scenarios suffers from a one-to-many mapping problem: a shared global representation entangling co-occurring objects is stored as class-wise cache prototypes, inducing dominant-label bias and compromised cache calibration. While introducing region-level cues helps isolate class-specific evidence, such regional evidence can also be unreliable under distribution shifts, making its identification and utilization non-trivial. To address these issues, we introduce PuRF, a novel PuRiFication-driven cache-based method for multi-label test-time adaptation of vision-language models. Specifically, PuRF first performs region purification to identify reliable regions, providing comprehensive regional cues for multi-label recognition and enabling fine-grained alignment. Based on these purified regions, PuRF conducts cache purification to enhance cache representation and adaptability, where episodic purification builds a discriminative region-based cache, and temporal refreshing further promotes long-term cache adaptability. Experiments demonstrate that PuRF consistently outperforms state-of-the-art methods, achieving a notable 4.05% mAP improvement on ViT-B/32 across five datasets.

Related papers

• Back to Source: Open-Set Continual Test-Time Adaptation via Domain Compensation [18.70132691409063]
  Test-Time Adaptation aims to mitigate distributional shifts between training and test domains during inference time.<n>The coupling of domain and semantic shifts often collapses the feature space, severely degrading both classification and out-of-distribution detection.<n>We propose DOmain COmpensation (DOCO), a lightweight and effective framework that robustly performs domain adaptation and OOD detection in a synergistic, closed loop.
  arXiv  Detail & Related papers  (2026-04-23T15:29:29Z)
• Test-Time Adaptation via Cache Personalization for Facial Expression Recognition in Videos [59.83490704563065]
  This paper introduces Cache Personalization (TTA-CaP), a cache-based TTA method that enables cost-effective personalization of vision-language models for video FER.<n>Our experiments show TTA-CaP can outperform state-of-the-art TTA methods under subject-specific and environmental shifts.
  arXiv  Detail & Related papers  (2026-03-22T16:31:25Z)
• DiCache: Let Diffusion Model Determine Its Own Cache [62.954717254728166]
  DiCache is a training-free adaptive caching strategy for accelerating diffusion models at runtime.<n>Online Probe Profiling Scheme leverages a shallow-layer online probe to obtain an on-the-fly indicator for the caching error in real time.<n> Dynamic Cache Trajectory Alignment approximates the deep-layer feature output from multi-step historical caches.
  arXiv  Detail & Related papers  (2025-08-24T13:30:00Z)
• Advancing Reliable Test-Time Adaptation of Vision-Language Models under Visual Variations [67.35596444651037]
  Vision-language models (VLMs) exhibit remarkable zero-shot capabilities but struggle with distribution shifts in downstream tasks when labeled data is unavailable.<n>We propose a Reliable Test-time Adaptation (ReTA) method that enhances reliability from two perspectives.
  arXiv  Detail & Related papers  (2025-07-13T05:37:33Z)
• ADAptation: Reconstruction-based Unsupervised Active Learning for Breast Ultrasound Diagnosis [11.49367029555765]
  Deep learning-based diagnostic models often suffer performance drops due to distribution shifts between training (source) and test (target) domains.<n>We propose a novel unsupervised Active learning framework for Adaptation Domain, named ADAptation.<n>Our method efficiently selects informative samples from multi-domain data pools under limited annotation budget.
  arXiv  Detail & Related papers  (2025-07-01T06:45:02Z)
• Mitigating Cache Noise in Test-Time Adaptation for Large Vision-Language Models [13.157596316463621]
  Test-time adaptation (TTA) of visual language models has attracted significant attention as a solution to the performance degradation caused by distribution shifts in downstream tasks.<n>We introduce a comprehensive and reliable caching mechanism and propose a novel zero-shot TTA method called "Cache, Residual, Gaussian" (CRG)<n> Experimental results on 13 benchmarks demonstrate that CRG outperforms state-of-the-art TTA methods, showcasing exceptional robustness and adaptability.
  arXiv  Detail & Related papers  (2025-03-24T04:32:35Z)
• DOTA: Distributional Test-Time Adaptation of Vision-Language Models [69.41389326333771]
  Vision-language foundation models can be unreliable when significant distribution gaps exist between training and test data.<n>We propose DOTA (DistributiOnal Test-time Adaptation), a simple yet effective method addressing this limitation.<n>This distribution-centric approach enables the model to continually learn and adapt to the deployment environment.
  arXiv  Detail & Related papers  (2024-09-28T15:03:28Z)
• Enhancing Test Time Adaptation with Few-shot Guidance [62.49199492255226]
  Deep neural networks often encounter significant performance drops while facing with domain shifts between training (source) and test (target) data.<n>Test Time Adaptation (TTA) methods have been proposed to adapt pre-trained source model to handle out-of-distribution streaming target data.<n>We develop Few-Shot Test Time Adaptation (FS-TTA), a novel and practical setting that utilizes a few-shot support set on top of TTA.
  arXiv  Detail & Related papers  (2024-09-02T15:50:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.