FACTOR: Counterfactual Training-Free Test-Time Adaptation for Open-Vocabulary Object Detection

• URL: http://arxiv.org/abs/2605.03294v1
• Date: Tue, 05 May 2026 02:31:18 GMT
• Title: FACTOR: Counterfactual Training-Free Test-Time Adaptation for Open-Vocabulary Object Detection
• Abstract summary: FACTOR is a training-free Test-time adaptation for Open-vocabulary object detection.<n>It quantifies attribute sensitivity, semantic relevance, and prediction variation to selectively suppress attribute-dependent predictions.<n>Experiments on PASCAL-C, COCO-C, and FoggyCityscapes show that FACTOR consistently outperforms prior TTA methods.
• Score: 63.91351553178842
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open-vocabulary object detection often fails under distribution shifts, as it can be misled by spurious correlations between non-causal visual attributes (e.g., brightness, texture) and object categories. Existing test-time adaptation (TTA) methods either depend on costly online optimization or perform global calibration, overlooking the attribute-specific nature of these failures. To address this, we propose FACTOR (counterFACtual training-free Test-time adaptation for Open-vocabulaRy object detection), a lightweight framework grounded in counterfactual reasoning. By perturbing test images along non-causal attributes and comparing region-level predictions between original and counterfactual views, FACTOR quantifies attribute sensitivity, semantic relevance, and prediction variation to selectively suppress attribute-dependent predictions-without parameter updates. Experiments on PASCAL-C, COCO-C, and FoggyCityscapes show that FACTOR consistently outperforms prior TTA methods, demonstrating that explicit counterfactual reasoning effectively improves robustness under distribution shifts.

Related papers

• PISA: A Pseudo-Individual Source-Domain Feature Adaptation Framework for Test-Time Open-Vocabulary Object Detection [3.403013334650533]
  PISA is a source-free OVOD-TTA method seamlessly integrated into open-vocabulary visual backbones.<n>We show PISA substantially improves both the localization precision and the category recognition accuracy of the original models.
  arXiv  Detail & Related papers  (2026-08-14T09:47:44Z)
• Robustifying Vision-Language Models via Test-Time Prompt Adaptation [23.05168102474528]
  We propose RITA, a test-tIme prompt-TAdaptation framework that shifts from sample-level estimates to distribution-level alignment.<n>Specifically, RITA employs optimal transport to align the distribution of augmented visual features with textual prototypes, mitigating adversarial outliers and rectifying cross-modal semantic misalignment.
  arXiv  Detail & Related papers  (2026-07-10T14:19:42Z)
• AEGIS: A Semantic GAN and Evidential Learning Frameworkfor Robust Adversarial Detection in Vision Sensors [11.468095590140777]
  AEGIS is a semantic aware and uncertainty guided adversarial detection framework for vision sensors pipelines.<n>A SemantiGAN module functions as a multi class semantic discriminator, identifying and filtering visually inconsistent adversarial inputs.<n> Evaluations on the Tiny ImageNet dataset demonstrate the effectiveness of AEGIS.
  arXiv  Detail & Related papers  (2026-06-25T12:03:02Z)
• MoASE++: Mixture of Activation Sparsity Experts with Domain-Adaptive On-policy Distillation for Continual Test Time Adaptation [65.068801413044]
  Continual test-time adaptation adapts a source-pretrained model to non-stationary, unlabeled target streams.<n>We introduce MoASE, a mixture-of-experts that disentangles domain-agnostic structure from domain-specific texture.
  arXiv  Detail & Related papers  (2026-05-18T01:52:12Z)
• Reward-Guided Semantic Evolution for Test-time Adaptive Object Detection [82.2968697030677]
  Open-vocabulary object detection with vision-language models (VLMs) such as Grounding DINO suffers from performance degradation under test-time distribution shifts.<n>We propose Reward-Guided Semantic Evolution (RGSE), a training-free framework that directly refines the text embeddings at test time.
  arXiv  Detail & Related papers  (2026-05-06T06:17:41Z)
• Fair Context Learning for Evidence-Balanced Test-Time Adaptation in Vision-Language Models [10.45965859391796]
  Test-Time Adaptation (TTA) aims to improve robustness using only unlabeled test samples.<n>Most prompt-based TTA methods rely on entropy minimization.<n>We propose Fair Context Learning (FCL) that avoids entropy minimization by explicitly addressing shared-evidence bias.
  arXiv  Detail & Related papers  (2026-02-02T16:02:50Z)
• Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples [11.727747752958436]
  Deep learning models often learn features that spuriously correlate with the class label during training but are irrelevant to the prediction task.<n>Existing methods typically address this issue by annotating potential spurious attributes, or filtering spurious features based on some empirical assumptions.<n>We propose a data-oriented approach to mitigate the spurious correlation in deep learning models.
  arXiv  Detail & Related papers  (2025-12-28T10:54:51Z)
• Test-Time Adaptation by Causal Trimming [25.388362454409833]
  Test-time adaptation aims to improve model robustness under distribution shifts.<n>We introduce Test-time Adaptation by Causal Trimming (TACT)<n>TACT applies data augmentations that preserve causal features while varying non-causal ones.<n>It consistently outperforms state-of-the-art methods by a significant margin.
  arXiv  Detail & Related papers  (2025-10-13T08:22:38Z)
• Bayesian Test-time Adaptation for Object Recognition and Detection with Vision-language Models [86.53246292425699]
  We present BCA+, a training-free framework for TTA for both object recognition and detection.<n>We formulate adaptation as a Bayesian inference problem, where final predictions are generated by fusing the initial VLM output with a cache-based prediction.<n>BCA+ achieves state-of-the-art performance on both recognition and detection benchmarks.
  arXiv  Detail & Related papers  (2025-10-03T06:27:33Z)
• TRUST: Test-time Resource Utilization for Superior Trustworthiness [15.031121920821109]
  We propose a novel test-time optimization method that accounts for the impact of such noise to produce more reliable confidence estimates.<n>This score defines a monotonic subset-selection function, where population accuracy consistently increases as samples with lower scores are removed.
  arXiv  Detail & Related papers  (2025-06-06T12:52:32Z)
• Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting [65.21599711087538]
  Test-time adaptation (TTA) seeks to tackle potential distribution shifts between training and test data by adapting a given model w.r.t. any test sample.<n>Prior methods perform backpropagation for each test sample, resulting in unbearable optimization costs to many applications.<n>We propose an Efficient Anti-Forgetting Test-Time Adaptation (EATA) method which develops an active sample selection criterion to identify reliable and non-redundant samples.
  arXiv  Detail & Related papers  (2024-03-18T05:49:45Z)
• Model-based causal feature selection for general response types [8.228587135343071]
  Invariant causal prediction (ICP) is a method for causal feature selection which requires data from heterogeneous settings. We develop transformation-model (TRAM) based ICP, allowing for continuous, categorical, count-type, and uninformatively censored responses. We provide an open-source R package 'tramicp' and evaluate our approach on simulated data and in a case study investigating causal features of survival in critically ill patients.
  arXiv  Detail & Related papers  (2023-09-22T12:42:48Z)
• Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognition [65.84978547406753]
  Test-time Adaptation aims to adapt the model trained on source domains to yield better predictions for test samples. Single-Utterance Test-time Adaptation (SUTA) is the first TTA study in speech area to our best knowledge.
  arXiv  Detail & Related papers  (2022-03-27T06:38:39Z)
• Training on Test Data with Bayesian Adaptation for Covariate Shift [96.3250517412545]
  Deep neural networks often make inaccurate predictions with unreliable uncertainty estimates. We derive a Bayesian model that provides for a well-defined relationship between unlabeled inputs under distributional shift and model parameters. We show that our method improves both accuracy and uncertainty estimation.
  arXiv  Detail & Related papers  (2021-09-27T01:09:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.