VisPlay: Self-Evolving Vision-Language Models from Images

• URL: http://arxiv.org/abs/2511.15661v2
• Date: Thu, 20 Nov 2025 12:43:54 GMT
• Title: VisPlay: Self-Evolving Vision-Language Models from Images
• Abstract summary: Reinforcement learning (RL) provides a principled framework for improving Vision-Language Models (VLMs) on complex tasks.<n>We introduce VisPlay, a self-evolving RL framework that enablesVLMs to autonomously improve their reasoning abilities using large amounts of unlabeled image data.
• Score: 16.43670977857598
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning (RL) provides a principled framework for improving Vision-Language Models (VLMs) on complex reasoning tasks. However, existing RL approaches often rely on human-annotated labels or task-specific heuristics to define verifiable rewards, both of which are costly and difficult to scale. We introduce VisPlay, a self-evolving RL framework that enables VLMs to autonomously improve their reasoning abilities using large amounts of unlabeled image data. Starting from a single base VLM, VisPlay assigns the model into two interacting roles: an Image-Conditioned Questioner that formulates challenging yet answerable visual questions, and a Multimodal Reasoner that generates silver responses. These roles are jointly trained with Group Relative Policy Optimization (GRPO), which incorporates diversity and difficulty rewards to balance the complexity of generated questions with the quality of the silver answers. VisPlay scales efficiently across two model families. When trained on Qwen2.5-VL and MiMo-VL, VisPlay achieves consistent improvements in visual reasoning, compositional generalization, and hallucination reduction across eight benchmarks, including MM-Vet and MMMU, demonstrating a scalable path toward self-evolving multimodal intelligence. The project page is available at https://bruno686.github.io/VisPlay/

Related papers

• Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models [54.097381112138]
  Current self-evolving LMMs struggle on vision-language understanding tasks such as image captioning and visual question answering.<n>We propose VISE, a purely unsupervised self-evolving framework that directly regularizes the model's visual conditioning policy.<n>VISE operates within a single model without specialist roles, external reward models, or annotations.
  arXiv  Detail & Related papers  (2026-06-25T17:59:55Z)
• RISE: Reliable Improvement in Self-Evolving Vision-Language Models [22.16641554921884]
  Vision-language models (VLMs) have achieved strong multimodal reasoning capabilities.<n>VLMs still rely heavily on large-scale human-constructed supervision for post-training.<n>We propose textbfRISE, a reliable self-evolving framework for vision-language models.
  arXiv  Detail & Related papers  (2026-05-20T08:57:57Z)
• MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Data [30.295273919299046]
  We present Multi-model Multimodal Zero (MM-Zero), the first RL-based framework to achieve zero-data self-evolution for Vision Language Models.<n> MM-Zero introduces a multi-role self-evolving training framework comprising three specialized roles.<n>Our experiments show that MM-Zero improves VLM reasoning performance across a wide range of multimodal benchmarks.
  arXiv  Detail & Related papers  (2026-03-10T05:23:26Z)
• MVR: Multi-view Video Reward Shaping for Reinforcement Learning [17.20077949643041]
  Multi-View Video Reward Shaping (MVR) is a framework that models the relevance of states regarding the target task using videos captured from multiple viewpoints.<n>MVR learns a state relevance function that mitigates the bias towards specific static poses inherent in image-based methods.<n>We introduce a state-dependent reward shaping formulation that integrates task-specific rewards and VLM-based guidance.
  arXiv  Detail & Related papers  (2026-03-02T10:24:04Z)
• Guiding the Inner Eye: A Framework for Hierarchical and Flexible Visual Grounded Reasoning [6.800544911407401]
  GRiP (Guided Reasoning and Perception) is a novel training framework for visual grounded reasoning.<n> GRiP cultivates robust and flexible visual grounded reasoning by explicitly guiding the model's perceptual focus and logical pathways.<n> GRiP achieves state-of-the-art results among open-source models on the highly challenging TreeBench and V* Bench.
  arXiv  Detail & Related papers  (2025-11-27T07:18:25Z)
• EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewards [52.42920996842378]
  We propose a self-evolving framework, named EvoLMM, that instantiates two cooperative agents from a single backbone model.<n>This dynamic feedback encourages both the generation of informative queries and the refinement of structured reasoning.<n>Our code and models are available at https://github.com/mbzuai-oryx/EvoLMM.
  arXiv  Detail & Related papers  (2025-11-20T18:59:54Z)
• Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Models [63.69856480318313]
  AGILE formulates jigsaw solving as an interactive process, enabling the model to progressively engage with the environment.<n>We show that AGILE substantially boosts performance on jigsaw tasks of varying complexity.<n>We also demonstrate strong generalization across 9 general vision tasks, achieving an average improvement of 3.1%.
  arXiv  Detail & Related papers  (2025-10-01T17:58:05Z)
• Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play [27.15191616995172]
  Vision-Zero is a framework enabling vision-language models (VLMs) self-improvement through competitive visual games generated from arbitrary image pairs.<n> Vision-Zero trains VLMs in "Who Is the Spy"-style games, where the models engage in strategic reasoning and actions across multiple roles.<n>Unlike existing gamified frameworks, Vision-Zero can generate games from arbitrary images, thereby enhancing the model's reasoning ability across diverse domains.
  arXiv  Detail & Related papers  (2025-09-29T21:55:55Z)
• VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning [95.89543460132413]
  Vision-language models (VLMs) have improved performance by increasing the number of visual tokens.<n>However, most real-world scenarios do not require such an extensive number of visual tokens.<n>We present a new paradigm for visual token compression, namely, VisionThink.
  arXiv  Detail & Related papers  (2025-07-17T17:59:55Z)
• Demystifying the Visual Quality Paradox in Multimodal Large Language Models [49.154146792279946]
  Recent Multimodal Large Language Models (MLLMs) excel on benchmark vision-language tasks, yet little is known about how input visual quality shapes their responses.<n>We conduct the first systematic study spanning leading MLLMs and a suite of vision-language benchmarks.<n>We uncover a visual-quality paradox: model, task, and even individual-instance performance can improve when images deviate from human-perceived fidelity.
  arXiv  Detail & Related papers  (2025-06-18T17:14:07Z)
• Understand, Think, and Answer: Advancing Visual Reasoning with Large Multimodal Models [26.14137626882127]
  Large Multimodal Models (LMMs) have recently demonstrated remarkable visual understanding performance on both vision-language and vision-centric tasks.<n>We present a unified visual reasoning mechanism that enables LMMs to solve complicated compositional problems.<n>Our trained model, Griffon-R, has the ability of end-to-end automatic understanding, self-thinking, and reasoning answers.
  arXiv  Detail & Related papers  (2025-05-27T05:50:25Z)
• One RL to See Them All: Visual Triple Unified Reinforcement Learning [92.90120580989839]
  We propose V-Triune, a Visual Triple Unified Reinforcement Learning system that enables visual reasoning and perception tasks within a single training pipeline.<n>V-Triune comprises triple complementary components: Sample-Level Datashelf (to unify diverse task inputs), Verifier-Level Reward (to deliver custom rewards via specialized verifiers).<n>We introduce a novel Dynamic IoU reward, which provides adaptive, progressive, and definite feedback for perception tasks handled by V-Triune.
  arXiv  Detail & Related papers  (2025-05-23T17:41:14Z)
• Game-RL: Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoning [89.93384726755106]
  Vision-language reinforcement learning (RL) has primarily focused on narrow domains.<n>We find video games inherently provide rich visual elements and mechanics that are easy to verify.<n>To fully use the multimodal and verifiable reward in video games, we propose Game-RL.
  arXiv  Detail & Related papers  (2025-05-20T03:47:44Z)
• What is the Visual Cognition Gap between Humans and Multimodal LLMs? [63.81347276258992]
  We evaluate the visual cognition capability of Multimodal Large Language Models (MLLMs) and compare their performance with human visual cognition studies.<n>Our comparative experiments with different baselines reveal a gap between MLLMs and human intelligence.<n>We believe that the public release of MaRs-VQA and the Qwen2-VCog baseline model will drive progress toward the next generation of MLLMs with human-like visual cognition abilities.
  arXiv  Detail & Related papers  (2024-06-14T22:02:21Z)
• Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models [81.71651422951074]
  Chain-of-Spot (CoS) method is a novel approach that enhances feature extraction by focusing on key regions of interest. This technique allows LVLMs to access more detailed visual information without altering the original image resolution. Our empirical findings demonstrate a significant improvement in LVLMs' ability to understand and reason about visual content.
  arXiv  Detail & Related papers  (2024-03-19T17:59:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.