GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models

• URL: http://arxiv.org/abs/2507.11969v1
• Date: Wed, 16 Jul 2025 07:02:45 GMT
• Title: GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
• Abstract summary: Global-Spatial Bias Learner (GS-Bias) is an efficient and effective TTA paradigm that incorporates two learnable biases during TTA.<n>GS-Bias achieves high efficiency while achieving state-of-the-art performance on 15 benchmark datasets.
• Score: 49.598380958154706
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in test-time adaptation (TTA) for Vision-Language Models (VLMs) have garnered increasing attention, particularly through the use of multiple augmented views of a single image to boost zero-shot generalization. Unfortunately, existing methods fail to strike a satisfactory balance between performance and efficiency, either due to excessive overhead of tuning text prompts or unstable benefits from handcrafted, training-free visual feature enhancement. In this paper, we present Global-Spatial Bias Learner (GS-Bias), an efficient and effective TTA paradigm that incorporates two learnable biases during TTA, unfolded as the global bias and spatial bias. Particularly, the global bias captures the global semantic features of a test image by learning consistency across augmented views, while spatial bias learns the semantic coherence between regions in the image's spatial visual representation. It is worth highlighting that these two sets of biases are directly added to the logits outputed by the pretrained VLMs, which circumvent the full backpropagation through VLM that hinders the efficiency of existing TTA methods. This endows GS-Bias with extremely high efficiency while achieving state-of-the-art performance on 15 benchmark datasets. For example, it achieves a 2.23% improvement over TPT in cross-dataset generalization and a 2.72% improvement in domain generalization, while requiring only 6.5% of TPT's memory usage on ImageNet.

Related papers

• Test-Time Augmentation for Tabular-to-Image Classifiers under Distribution Shifts [2.1272291601429116]
  Test-Time Augmentation (TTA) is an effective approach in image classification to improve model robustness.<n>This work evaluates the impact of TTA techniques on predictive performance under Out-Of-Distribution (OOD) for representations generated by tabular-to-image methods.<n>The results indicate that TTA improves OOD performance, with composite and photometric strategies providing the best trade-off between robustness and variance.
  arXiv  Detail & Related papers  (2026-08-04T12:26:58Z)
• Exposing Hidden Biases in Text-to-Image Models via Automated Prompt Search [18.386455479798574]
  Bias-Guided Prompt Search (BGPS) is a framework that automatically generates prompts that aim to maximize the presence of biases in images.<n>BGPS comprises two components: (1) an LLM instructed to produce attribute-neutral prompts and (2) attribute classifiers acting on the TTI's internal representations.<n>We conduct extensive experiments on Stable Diffusion 1.5 and a state-of-the-art debiased model and discover an array of subtle and previously undocumented biases.
  arXiv  Detail & Related papers  (2025-12-09T15:39:04Z)
• Unleashing the Power of Vision-Language Models for Long-Tailed Multi-Label Visual Recognition [55.189113121465816]
  We propose a novel correlation adaptation prompt network (CAPNET) for long-tailed multi-label visual recognition.<n>CAPNET explicitly models correlations from CLIP's textual encoder.<n>It improves generalization through test-time ensembling and realigns visual-textual modalities.
  arXiv  Detail & Related papers  (2025-11-25T18:57:28Z)
• Towards Reliable and Holistic Visual In-Context Learning Prompt Selection [82.23704441763651]
  Visual In-Context Learning (VICL) has emerged as a prominent approach for adapting visual foundation models to novel tasks.<n>VICL methods, such as Partial2Global and VPR, are grounded in the similarity-priority assumption that images more visually similar to a query image serve as better in-context examples.<n>This paper introduces an enhanced variant of Partial2Global designed for reliable and holistic selection of in-context examples in VICL.
  arXiv  Detail & Related papers  (2025-09-30T09:23:12Z)
• Simple yet Effective Semi-supervised Knowledge Distillation from Vision-Language Models via Dual-Head Optimization [47.38380084735716]
  vision-supervised models (VLMs), pre-trained on massive image-text pairs, have demonstrated remarkable zero-/few-shot performance.<n>Knowledge distillation (KD) offers a natural framework for transferring VLM capabilities, but it suffers from gradient conflicts between supervised and distillation losses.<n>We propose Dual-Head Optimization (DHO), which introduces dual prediction heads for each distinct signal.
  arXiv  Detail & Related papers  (2025-05-12T15:39:51Z)
• EAM: Enhancing Anything with Diffusion Transformers for Blind Super-Resolution [11.331361804059625]
  Enhancing Anything Model (EAM) is a novel Blind Super-Resolution method.<n>We introduce a novel block, $Psi$-DiT, which effectively guides the DiT to enhance image restoration.<n>EAM achieves state-of-the-art results across multiple datasets, outperforming existing methods in both quantitative metrics and visual quality.
  arXiv  Detail & Related papers  (2025-05-08T13:03:07Z)
• Search is All You Need for Few-shot Anomaly Detection [39.737510049667556]
  Few-shot anomaly detection (FSAD) has emerged as a crucial yet challenging task in industrial inspection.<n>We show that a straightforward nearest-neighbor search framework can surpass state-of-the-art performance in both single-class and multi-class FSAD scenarios.<n>Our method achieves remarkable image-level AUROC scores of 97.4%, 94.8%, and 70.8% respectively.
  arXiv  Detail & Related papers  (2025-04-16T09:21:34Z)
• Separate-and-Enhance: Compositional Finetuning for Text2Image Diffusion Models [58.46926334842161]
  This work illuminates the fundamental reasons for such misalignment, pinpointing issues related to low attention activation scores and mask overlaps. We propose two novel objectives, the Separate loss and the Enhance loss, that reduce object mask overlaps and maximize attention scores. Our method diverges from conventional test-time-adaptation techniques, focusing on finetuning critical parameters, which enhances scalability and generalizability.
  arXiv  Detail & Related papers  (2023-12-10T22:07:42Z)
• X-Transfer: A Transfer Learning-Based Framework for GAN-Generated Fake Image Detection [33.31312811230408]
  misuse of GANs for generating deceptive images, such as face replacement, raises significant security concerns. This paper introduces a novel GAN-generated image detection algorithm called X-Transfer. It enhances transfer learning by utilizing two neural networks that employ interleaved parallel gradient transmission.
  arXiv  Detail & Related papers  (2023-10-07T01:23:49Z)
• DeAR: Debiasing Vision-Language Models with Additive Residuals [5.672132510411465]
  Large pre-trained vision-language models (VLMs) provide rich, adaptable image and text representations. These models suffer from societal biases owing to the skewed distribution of various identity groups in the training data. We present DeAR, a novel debiasing method that learns additive residual image representations to offset the original representations.
  arXiv  Detail & Related papers  (2023-03-18T14:57:43Z)
• Anticipating the Unseen Discrepancy for Vision and Language Navigation [63.399180481818405]
  Vision-Language Navigation requires the agent to follow natural language instructions to reach a specific target. The large discrepancy between seen and unseen environments makes it challenging for the agent to generalize well. We propose Unseen Discrepancy Anticipating Vision and Language Navigation (DAVIS) that learns to generalize to unseen environments via encouraging test-time visual consistency.
  arXiv  Detail & Related papers  (2022-09-10T19:04:40Z)
• Reducing the Vision and Language Bias for Temporal Sentence Grounding [22.571577672704716]
  We propose a Debiasing-TSG (D-TSG) model to filter and remove the negative biases in both vision and language modalities. We demonstrate its effectiveness by achieving the state-of-the-art performance on three benchmark datasets.
  arXiv  Detail & Related papers  (2022-07-27T11:18:45Z)
• Self-Promoted Supervision for Few-Shot Transformer [178.52948452353834]
  Self-promoted sUpervisioN (SUN) is a few-shot learning framework for vision transformers (ViTs) SUN pretrains the ViT on the few-shot learning dataset and then uses it to generate individual location-specific supervision for guiding each patch token. Experiments show that SUN using ViTs significantly surpasses other few-shot learning frameworks with ViTs and is the first one that achieves higher performance than those CNN state-of-the-arts.
  arXiv  Detail & Related papers  (2022-03-14T12:53:27Z)
• Dense Contrastive Visual-Linguistic Pretraining [53.61233531733243]
  Several multimodal representation learning approaches have been proposed that jointly represent image and text. These approaches achieve superior performance by capturing high-level semantic information from large-scale multimodal pretraining. We propose unbiased Dense Contrastive Visual-Linguistic Pretraining to replace the region regression and classification with cross-modality region contrastive learning.
  arXiv  Detail & Related papers  (2021-09-24T07:20:13Z)
• To be Critical: Self-Calibrated Weakly Supervised Learning for Salient Object Detection [95.21700830273221]
  Weakly-supervised salient object detection (WSOD) aims to develop saliency models using image-level annotations. We propose a self-calibrated training strategy by explicitly establishing a mutual calibration loop between pseudo labels and network predictions. We prove that even a much smaller dataset with well-matched annotations can facilitate models to achieve better performance as well as generalizability.
  arXiv  Detail & Related papers  (2021-09-04T02:45:22Z)
• Calibrating Class Activation Maps for Long-Tailed Visual Recognition [60.77124328049557]
  We present two effective modifications of CNNs to improve network learning from long-tailed distribution. First, we present a Class Activation Map (CAMC) module to improve the learning and prediction of network classifiers. Second, we investigate the use of normalized classifiers for representation learning in long-tailed problems.
  arXiv  Detail & Related papers  (2021-08-29T05:45:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.