Harnessing Image Question Dependence for Better VLM Test-time Reinforcement Learning

• URL: http://arxiv.org/abs/2609.13296v1
• Date: Wed, 09 Sep 2026 22:54:15 GMT
• Title: Harnessing Image Question Dependence for Better VLM Test-time Reinforcement Learning
• Abstract summary: We propose TTIQ, a test-time reinforcement learning framework that harnesses image-question dependence for better vlm adaptation.<n> TTIQ teacher-forces each sampled response under the original image-question pair and its image- and question-ablated variants.<n>It combines image and question dependence with confidence to construct a response-level reward that favors jointly grounded responses.
• Score: 66.13367818892503
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Test-time reinforcement learning can adapt vision-language models (VLMs) to unlabeled target data, but its effectiveness is fundamentally limited by the reliability of self-generated learning signals. To assess the reliability of consensus-based learning signals, we analyze VLM test-time reinforcement learning across diverse VQA datasets and model sizes, revealing two limitations. First, gains from consensus-based test-time training largely come from answer normalization rather than content correction. Second, many initial VLM responses are incorrect due to the model's limited ability to jointly use the image and the question; consensus rewards derived from these outputs may preserve the resulting grounding errors rather than correct them. Motivated by these, we propose TTIQ, a test-time reinforcement learning framework that harnesses image-question dependence for better vlm adaptation. TTIQ teacher-forces each sampled response under the original image-question pair and its image- and question-ablated variants, using the resulting token-level likelihood changes to estimate dependence on each input. It combines image and question dependence with calibrated confidence to construct a response-level reward that favors jointly grounded responses, and uses the token-level signals to assign greater positive policy credit to tokens supported by both inputs. This design favors responses that are jointly grounded in the image and the question and sufficiently confident, rather than merely popular. Experiments across eight VQA datasets and multiple VLM sizes show that TTIQ achieves the best average performance at every model scale. It further generalizes across VLM families, while models trained on one dataset improve performance on unseen datasets without further training.

Related papers

• AVSD: Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals [65.71225905458182]
  Self-distillation enables language models to learn on-policy from their own trajectories by using the same model as both student and teacher.<n>This setup provides dense token-level feedback without relying on a separate external model.<n>We introduce AVSD, a novel method of self-distillation with multiple privileged-information views.
  arXiv  Detail & Related papers  (2026-05-20T03:06:36Z)
• TTA-Vid: Generalized Test-Time Adaptation for Video Reasoning [54.70019148172847]
  We leverage the paradigm of Test-Time Reinforcement Learning on video-language data to allow for adapting a pretrained model to incoming video samples at test-time without explicit labels.<n>The proposed test-time adaptation for video approach (TTA-Vid) combines two components that work simultaneously.<n>TTA-Vid yields consistent improvements across various video reasoning tasks and is able to outperform current state-of-the-art methods trained on large-scale data.
  arXiv  Detail & Related papers  (2026-04-01T09:52:57Z)
• Continuous Self-Improvement of Large Language Models by Test-time Training with Verifier-Driven Sample Selection [6.471199527741301]
  We introduce a new framework called VDS-TTT - Verifier-Driven Sample Selection for Test-Time Training.<n>We use a learned verifier to score a pool of generated responses and select only from high ranking pseudo-labeled examples for fine-tuned adaptation.<n>We fine-tune only low-rank LoRA adapter parameters, ensuring adaptation efficiency and fast convergence.
  arXiv  Detail & Related papers  (2025-05-26T03:54:47Z)
• QIRL: Boosting Visual Question Answering via Optimized Question-Image Relation Learning [38.24779287568123]
  Current debiasing techniques fail to capture the superior relation between images and texts.<n>No prior work has examined the degree of input relevance in debiasing studies.<n>We propose a novel framework, which employs a generation-based self-supervised learning strategy.
  arXiv  Detail & Related papers  (2025-04-04T10:38:28Z)
• Teaching LMMs for Image Quality Scoring and Interpreting [71.1335005098584]
  We propose Q-SiT (Quality Scoring and Interpreting joint Teaching), a unified framework that enables image quality scoring and interpreting simultaneously.<n>Q-SiT is the first model capable of simultaneously performing image quality scoring and interpreting tasks, along with its lightweight variant, Q-SiT-mini.<n> Experimental results demonstrate that Q-SiT achieves strong performance in both tasks with superior generalization IQA abilities.
  arXiv  Detail & Related papers  (2025-03-12T09:39:33Z)
• Graph-based Confidence Calibration for Large Language Models [22.394717844099684]
  We propose using an auxiliary learning model to assess response correctness based on the self-consistency of multiple outputs generated by the large language models.<n>Our method builds a consistency graph to represent the agreement among multiple responses and uses a graph neural network (GNN) to estimate the likelihood that each response is correct.
  arXiv  Detail & Related papers  (2024-11-03T20:36:44Z)
• Multi-Modal Prompt Learning on Blind Image Quality Assessment [65.0676908930946]
  Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly. Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness. Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings. This paper introduces an innovative multi-modal prompt-based methodology for IQA.
  arXiv  Detail & Related papers  (2024-04-23T11:45:32Z)
• CONVIQT: Contrastive Video Quality Estimator [63.749184706461826]
  Perceptual video quality assessment (VQA) is an integral component of many streaming and video sharing platforms. Here we consider the problem of learning perceptually relevant video quality representations in a self-supervised manner. Our results indicate that compelling representations with perceptual bearing can be obtained using self-supervised learning.
  arXiv  Detail & Related papers  (2022-06-29T15:22:01Z)
• Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalization [27.437077941786768]
  Vision-and-language (V&L) models pretrained on large-scale multimodal data have demonstrated strong performance on various tasks. We evaluate two pretrained V&L models under different settings by conducting cross-dataset evaluations. We find that these models tend to learn to solve the benchmark, rather than learning the high-level skills required by the VQA task.
  arXiv  Detail & Related papers  (2022-05-24T16:44:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.