GRPO-TTA: Test-Time Visual Tuning for Vision-Language Models via GRPO-Driven Reinforcement Learning

• URL: http://arxiv.org/abs/2605.03403v1
• Date: Tue, 05 May 2026 06:23:20 GMT
• Title: GRPO-TTA: Test-Time Visual Tuning for Vision-Language Models via GRPO-Driven Reinforcement Learning
• Abstract summary: Group Relative Policy Optimization (GRPO) has shown strong performance in post-training large language models and vision-language models.<n>We propose Group Relative Policy Optimization for Test-Time Adaptation (GRPO-TTA) to adapt GRPO to the TTA setting.<n>We show that GRPO-TTA consistently outperforms existing test-time adaptation methods.
• Score: 15.828209242407047
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Group Relative Policy Optimization (GRPO) has recently shown strong performance in post-training large language models and vision-language models. It raises a question of whether the GRPO also significantly promotes the test-time adaptation (TTA) of vision language models. In this paper, we propose Group Relative Policy Optimization for Test-Time Adaptation (GRPO-TTA), which adapts GRPO to the TTA setting by reformulating class-specific prompt prediction as a group-wise policy optimization problem. Specifically, we construct output groups by sampling top-K class candidates from CLIP similarity distributions, enabling probability-driven optimization without access to ground-truth labels. Moreover, we design reward functions tailored to test-time adaptation, including alignment rewards and dispersion rewards, to guide effective visual encoder tuning. Extensive experiments across diverse benchmarks demonstrate that GRPO-TTA consistently outperforms existing test-time adaptation methods, with notably larger performance gains under natural distribution shifts.

Related papers

• RRPO: Reference-Relative Policy Optimization with Stratified Conditional Rollouts [84.20747697259326]
  Group Relative Policy Optimization (GRPO) has shown strong effectiveness in reinforcement learning from verifiable feedback.<n>We propose textbfReference-Relative Policy Optimization (RRPO), which generalizes GRPO by replacing direct correctness-based advantage construction with reference-relative contrastive comparisons.<n>We evaluate RRPO using anchor-based contrastive advantages throughout policy optimization, without relying on task ground-truth verifiers.
  arXiv  Detail & Related papers  (2026-07-20T19:40:25Z)
• AdaGRPO: A Capability-Aware Adaptive Enhancement for Flow-based GRPO [78.36537400975298]
  Group Relative Policy Optimization (GRPO) has demonstrated remarkable success in aligning text-to-image (T2I) flow models with human preferences.<n>We have identified that the learning loop of current flow-based GRPO is fundamentally decoupled from the learner's current capability.<n>We propose Adaptive GRPO (AdaGRPO), a novel capability-aware RL algorithm tailored for flow models.
  arXiv  Detail & Related papers  (2026-06-05T02:07:08Z)
• Personalized Group Relative Policy Optimization for Heterogenous Preference Alignment [13.085014101959118]
  We introduce Personalized GRPO, a novel alignment framework that decouples advantage estimation from batch statistics.<n>We evaluate P-GRPO across diverse tasks and find that it consistently achieves faster convergence and higher rewards than standard GRPO.
  arXiv  Detail & Related papers  (2026-02-17T19:00:43Z)
• iGRPO: Self-Feedback-Driven LLM Reasoning [88.83313431248473]
  Large Language Models (LLMs) have shown promise in solving complex mathematical problems, yet they still fall short of producing accurate and consistent solutions.<n>We introduce Iterative Group Relative Policy Optimization (iGRPO), a two-stage extension of GRPO that adds dynamic self-conditioning through model-generated drafts.<n>Under matched rollout budgets, iGRPO consistently outperforms GRPO across base models.
  arXiv  Detail & Related papers  (2026-02-09T18:45:11Z)
• TL-GRPO: Turn-Level RL for Reasoning-Guided Iterative Optimization [97.18886232580131]
  Large language models have demonstrated strong reasoning capabilities in complex tasks through tool integration.<n>We propose Turn-Level GRPO, a lightweight RL algorithm that performs turn-level group sampling for fine-grained optimization.
  arXiv  Detail & Related papers  (2026-01-23T06:21:33Z)
• GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization [133.27496265096445]
  We show how to apply Group Relative Policy Optimization under multi-reward setting without examining its suitability.<n>We then introduce Group reward-Decoupled Normalization Policy Optimization (GDPO), a new policy optimization method to resolve these issues.<n>GDPO consistently outperforms GRPO, demonstrating its effectiveness and generalizability for multi-reward reinforcement learning optimization.
  arXiv  Detail & Related papers  (2026-01-08T18:59:24Z)
• GRPO-RM: Fine-Tuning Representation Models via GRPO-Driven Reinforcement Learning [52.16150076582931]
  We propose Group Relative Policy Optimization for Representation Model (GRPO-RM)<n>Our method establishes a predefined output set to functionally replace token sequence sampling in large language models (LLMs)<n>A specialized reward function is designed to accommodate the properties of representation models.
  arXiv  Detail & Related papers  (2025-11-19T09:19:39Z)
• VerIPO: Cultivating Long Reasoning in Video-LLMs via Verifier-Gudied Iterative Policy Optimization [59.39976343879587]
  VerIPO aims to gradually improve video LLMs' capacity for generating deep, long-term reasoning chains.<n>The training loop benefits from GRPO's expansive search and DPO's targeted optimization.<n>Our trained models exceed the direct inference of large-scale instruction-tuned Video-LLMs.
  arXiv  Detail & Related papers  (2025-05-25T06:41:28Z)
• Preference Alignment Improves Language Model-Based TTS [76.70693823683091]
  preference alignment algorithms adjust LMs to align with the preferences of reward models, enhancing the desirability of the generated content. With a 1.15B parameter LM-based TTS model, we demonstrate that preference alignment consistently improves intelligibility, speaker similarity, and proxy subjective evaluation scores.
  arXiv  Detail & Related papers  (2024-09-19T01:58:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.