OpenVLThinkerV2: A Generalist Multimodal Reasoning Model for Multi-domain Visual Tasks

• URL: http://arxiv.org/abs/2604.08539v1
• Date: Thu, 09 Apr 2026 17:59:39 GMT
• Title: OpenVLThinkerV2: A Generalist Multimodal Reasoning Model for Multi-domain Visual Tasks
• Abstract summary: We present a novel RL training objective that replaces standard linear scaling with non-linear distributional matching.<n>We also present OpenVLThinkerV2, a highly robust, general-purpose multimodal model.
• Score: 81.07598709704628
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Group Relative Policy Optimization (GRPO) has emerged as the de facto Reinforcement Learning (RL) objective driving recent advancements in Multimodal Large Language Models. However, extending this success to open-source multimodal generalist models remains heavily constrained by two primary challenges: the extreme variance in reward topologies across diverse visual tasks, and the inherent difficulty of balancing fine-grained perception with multi-step reasoning capabilities. To address these issues, we introduce Gaussian GRPO (G$^2$RPO), a novel RL training objective that replaces standard linear scaling with non-linear distributional matching. By mathematically forcing the advantage distribution of any given task to strictly converge to a standard normal distribution, $\mathcal{N}(0,1)$, G$^2$RPO theoretically ensures inter-task gradient equity, mitigates vulnerabilities to heavy-tail outliers, and offers symmetric update for positive and negative rewards. Leveraging the enhanced training stability provided by G$^2$RPO, we introduce two task-level shaping mechanisms to seamlessly balance perception and reasoning. First, response length shaping dynamically elicits extended reasoning chains for complex queries while enforce direct outputs to bolster visual grounding. Second, entropy shaping tightly bounds the model's exploration zone, effectively preventing both entropy collapse and entropy explosion. Integrating these methodologies, we present OpenVLThinkerV2, a highly robust, general-purpose multimodal model. Extensive evaluations across 18 diverse benchmarks demonstrate its superior performance over strong open-source and leading proprietary frontier models.

Related papers

• DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Models [55.01951088768769]
  DiffusionOPD is a new multi-task training paradigm for diffusion models based on Online Policy Distillation (OPD)<n>We show that DiffusionOPD consistently surpasses both multi-reward RL and cascade RL baselines in training efficiency and final performance.
  arXiv  Detail & Related papers  (2026-05-14T16:49:09Z)
• S-GRPO: Unified Post-Training for Large Vision-Language Models [11.901218794806796]
  We propose Supervised Group Relative Policy Optimization (S-GRPO) for adapting Large Vision-Language Models (LVLMs)<n>S-GRPO integrates the guidance of imitation learning into the multi-trajectory exploration of preference optimization.<n>It gracefully bridges the gap between Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL)
  arXiv  Detail & Related papers  (2026-04-17T08:39:07Z)
• Reinforcing Structured Chain-of-Thought for Video Understanding [49.3086326382764]
  We introduce Summary-Driven Reinforcement Learning (SDRL)<n>SDRL is a single-stage RL framework that obviates the need forSupervised Fine-Tuning (SFT)<n>Our method achieves state-of-the-art performance on seven public VideoQA datasets.
  arXiv  Detail & Related papers  (2026-03-26T22:11:14Z)
• OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL [63.388513841293616]
  Existing forgery detection methods fail to handle the interleaved text, images, and videos prevalent in real-world misinformation.<n>To bridge this gap, this paper targets to develop a unified framework for omnibus vision-language forgery detection and grounding.<n>We propose textbf OmniVL-Guard, a balanced reinforcement learning framework for omnibus vision-language forgery detection and grounding.
  arXiv  Detail & Related papers  (2026-02-11T09:41:36Z)
• TSPO: Breaking the Double Homogenization Dilemma in Multi-turn Search Policy Optimization [32.17940023097263]
  Multi-turn tool-integrated reasoning enables Large Language Models (LLMs) to solve complex tasks through iterative information retrieval.<n>Current reinforcement learning (RL) frameworks for search-augmented reasoning rely on sparse outcome-level rewards.<n>We propose Turn-level Stage-aware Policy Optimization (TSPO) to address this problem.
  arXiv  Detail & Related papers  (2026-01-30T09:58:45Z)
• Empowering Multi-Turn Tool-Integrated Reasoning with Group Turn Policy Optimization [20.004150645050537]
  Group Turn Policy Optimization (GTPO) is a novel reinforcement learning algorithm designed for training Large Language Models (LLMs) on multi-turn Tool-Integrated Reasoning tasks.<n>GTPO introduces three key innovations: turn-level reward assignment that provides fine-grained feedback for individual turns, return-based advantage estimation, and self-supervised reward shaping.<n>Our comprehensive evaluation demonstrates that GTPO outperforms GRPO by 3.0% on average across diverse reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-11-18T19:01:16Z)
• Adaptive Divergence Regularized Policy Optimization for Fine-tuning Generative Models [31.470613363668672]
  Adaptive Divergence Regularized Policy Optimization automatically adjusts regularization strength based on advantage estimates.<n>Our implementation with Wasserstein-2 regularization for flow matching generative models achieves remarkable results on text-to-image generation.<n> ADRPO generalizes to KL-regularized fine-tuning of both text-only LLMs and multi-modal reasoning models.
  arXiv  Detail & Related papers  (2025-10-20T19:46:02Z)
• Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn LLM Agents [28.145430029174577]
  Large language model (LLM)-based agents are increasingly trained with reinforcement learning (RL) to enhance their ability to interact with external environments.<n>Existing approaches typically rely on outcome-based rewards that are only provided at the final answer.<n>In this paper, we propose Information Gain-based Policy Optimization (IGPO), a simple yet effective RL framework that provides dense and intrinsic supervision for multi-turn agent training.
  arXiv  Detail & Related papers  (2025-10-16T17:59:32Z)
• G$^2$RPO: Granular GRPO for Precise Reward in Flow Models [74.21206048155669]
  We propose a novel Granular-GRPO (G$2$RPO) framework that achieves precise and comprehensive reward assessments of sampling directions.<n>We introduce a Multi-Granularity Advantage Integration module that aggregates advantages computed at multiple diffusion scales.<n>Our G$2$RPO significantly outperforms existing flow-based GRPO baselines.
  arXiv  Detail & Related papers  (2025-10-02T12:57:12Z)
• MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources [113.33902847941941]
  Variance-Aware Sampling (VAS) is a data selection strategy guided by Variance Promotion Score (VPS)<n>We release large-scale, carefully curated resources containing 1.6M long CoT cold-start data and 15k RL QA pairs.<n> Experiments across mathematical reasoning benchmarks demonstrate the effectiveness of both the curated data and the proposed VAS.
  arXiv  Detail & Related papers  (2025-09-25T14:58:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.