Mixture-of-Visual-Thoughts: Exploring Context-Adaptive Reasoning Mode Selection for General Visual Reasoning

• URL: http://arxiv.org/abs/2509.22746v1
• Date: Fri, 26 Sep 2025 04:33:53 GMT
• Title: Mixture-of-Visual-Thoughts: Exploring Context-Adaptive Reasoning Mode Selection for General Visual Reasoning
• Abstract summary: We propose a novel adaptive reasoning paradigm, Mixture-of-Visual-Thoughts (MoVT)<n>MoVT unifies different reasoning modes within a single model and guides it to select the appropriate mode based on context.<n>AdaVaR effectively guides the model to learn and differentiate multiple modes and perform context-adaptive mode selection.
• Score: 43.07723859242318
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current visual reasoning methods mainly focus on exploring specific reasoning modes. Although improvements can be achieved in particular domains, they struggle to develop general reasoning capabilities. Inspired by this, we propose a novel adaptive reasoning paradigm, Mixture-of-Visual-Thoughts (MoVT), which unifies different reasoning modes within a single model and guides it to select the appropriate mode based on context. To achieve this, we introduce AdaVaR, a two-stage Adaptive Visual Reasoning learning framework: different modes are unified and learned during the supervised cold-start stage, and the mode selection capability is induced via an RL process with a carefully designed AdaGRPO algorithm. Extensive experiments show that AdaVaR effectively guides the model to learn and differentiate multiple modes and perform context-adaptive mode selection, achieving consistent improvement across various scenarios, highlighting MoVT as an effective solution for building general visual reasoning models.

Related papers

• Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning [72.45536737928015]
  We propose Switch-Reasoner, a framework that learns to adaptively select reasoning modes for MLLMs.<n>It treats thinking as a virtual tool invocation and allows the model to either answer directly or invoke explicit reasoning before answering.<n>Experiments on 11 multimodal tasks show that Switch-Reasoner reduces unnecessary reasoning while maintaining strong performance.
  arXiv  Detail & Related papers  (2026-07-09T15:04:05Z)
• Policy Split: Incentivizing Dual-Mode Exploration in LLM Reinforcement with Dual-Mode Entropy Regularization [53.051213470905935]
  Policy Split is a novel paradigm that bifurcates the policy into normal and high-entropy modes with a high-entropy prompt.<n>While sharing model parameters, the two modes undergo collaborative dual-mode entropy regularization tailored to distinct objectives.
  arXiv  Detail & Related papers  (2026-04-13T14:13:06Z)
• AdaptMMBench: Benchmarking Adaptive Multimodal Reasoning for Mode Selection and Reasoning Process [35.95284812390557]
  We propose AdaptMMBench, a benchmark for adaptive multimodal reasoning across five domains: real-world, OCR, GUI, knowledge, and math.<n>Our evaluation reveals that while adaptive mode selection scales with model capacity, it notably decouples from final accuracy.
  arXiv  Detail & Related papers  (2026-02-02T19:00:27Z)
• MODE: Multi-Objective Adaptive Coreset Selection [24.64208190328432]
  mode adapts selection criteria to training phases: emphasizing class balance early, diversity during representation learning, and uncertainty at convergence.<n>We show that MODE achieves (1-1/e)-approximation with O(n log n) complexity and demonstrates competitive accuracy while providing interpretable insights into data utility evolution.
  arXiv  Detail & Related papers  (2025-12-24T12:43:40Z)
• Reasoning Palette: Modulating Reasoning via Latent Contextualization for Controllable Exploration for (V)LMs [49.66344956133349]
  Reasoning capacity shapes both inference-time performance and reinforcement learning (RL) training for large (vision-) language models.<n>This paper proposes Reasoning Palette, a novel latent-modulation framework that endows the model with a latent variable for strategic contextualization.
  arXiv  Detail & Related papers  (2025-12-19T03:32:53Z)
• Interleaved Latent Visual Reasoning with Selective Perceptual Modeling [42.93438443502933]
  Interleaved reasoning paradigms enhance Multimodal Large Language Models (MLLMs) with visual feedback but are hindered by the prohibitive computational cost.<n>A promising alternative, latent visual reasoning, circumvents this bottleneck yet currently forces a critical trade-off.<n>We introduce Interleaved Latent Visual Reasoning (ILVR), a framework that unifies dynamic state evolution with precise perceptual modeling.
  arXiv  Detail & Related papers  (2025-12-05T12:09:39Z)
• Omni-AutoThink: Adaptive Multimodal Reasoning via Reinforcement Learning [57.96134674544638]
  We propose a novel adaptive reasoning framework that dynamically adjusts the model's reasoning depth according to task difficulty.<n>Our framework comprises two stages: (1) an Adaptive Supervised Fine-Tuning stage, which endows the Omni model with fundamental reasoning capability using large-scale reasoning-augmented data, and (2) an Adaptive Reinforcement Learning stage, which optimize reasoning behaviors based on task complexity and reward feedback.
  arXiv  Detail & Related papers  (2025-12-03T13:33:28Z)
• Plug-and-Play Prompt Refinement via Latent Feedback for Diffusion Model Alignment [54.17386822940477]
  We introduce PromptLoop, a plug-and-play reinforcement learning framework that incorporates latent feedback into step-wise prompt refinement.<n>This design achieves a structural analogy to the Diffusion RL approach, while retaining the flexibility and generality of prompt-based alignment.
  arXiv  Detail & Related papers  (2025-10-01T02:18:58Z)
• Feature-Based vs. GAN-Based Learning from Demonstrations: When and Why [50.191655141020505]
  This survey provides a comparative analysis of feature-based and GAN-based approaches to learning from demonstrations.<n>We argue that the dichotomy between feature-based and GAN-based methods is increasingly nuanced.
  arXiv  Detail & Related papers  (2025-07-08T11:45:51Z)
• Variational Search Distributions [16.609027794680213]
  We develop VSD, a method for conditioning a generative model of discrete, variational designs on a rare desired class.<n>We empirically demonstrate that VSD can outperform existing baseline methods on a set of real sequence-design problems.
  arXiv  Detail & Related papers  (2024-09-10T01:33:31Z)
• Variance-Preserving-Based Interpolation Diffusion Models for Speech Enhancement [53.2171981279647]
  We present a framework that encapsulates both the VP- and variance-exploding (VE)-based diffusion methods. To improve performance and ease model training, we analyze the common difficulties encountered in diffusion models. We evaluate our model against several methods using a public benchmark to showcase the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-06-14T14:22:22Z)
• Trajectory-wise Multiple Choice Learning for Dynamics Generalization in Reinforcement Learning [137.39196753245105]
  We present a new model-based reinforcement learning algorithm that learns a multi-headed dynamics model for dynamics generalization. We incorporate context learning, which encodes dynamics-specific information from past experiences into the context latent vector. Our method exhibits superior zero-shot generalization performance across a variety of control tasks, compared to state-of-the-art RL methods.
  arXiv  Detail & Related papers  (2020-10-26T03:20:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.