When Seeing Is not Enough: Revealing the Limits of Active Reasoning in MLLMs

• URL: http://arxiv.org/abs/2510.15421v1
• Date: Fri, 17 Oct 2025 08:17:27 GMT
• Title: When Seeing Is not Enough: Revealing the Limits of Active Reasoning in MLLMs
• Authors: Hongcheng Liu, Pingjie Wang, Yuhao Wang, Siqu Ou, Yanfeng Wang, Yu Wang,
• Abstract summary: Multimodal large language models (MLLMs) have shown strong capabilities across a broad range of benchmarks.<n>Most existing evaluations focus on passive inference, where models perform step-by-step reasoning under complete information.<n>This raises a fundamental question: Can MLLMs actively acquire missing evidence under incomplete information?<n>We require the MLLMs to actively acquire missing evidence and iteratively refine decisions under incomplete information, by selecting a target image from a candidate pool without task-specific priors.<n>We evaluate 20 superior MLLMs and find that performance on active reasoning lags far behind it on passive settings, indicating substantial room for
• Score: 29.198301196459834
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal large language models (MLLMs) have shown strong capabilities across a broad range of benchmarks. However, most existing evaluations focus on passive inference, where models perform step-by-step reasoning under complete information. This setup is misaligned with real-world use, where seeing is not enough. This raises a fundamental question: Can MLLMs actively acquire missing evidence under incomplete information? To bridge this gap, we require the MLLMs to actively acquire missing evidence and iteratively refine decisions under incomplete information, by selecting a target image from a candidate pool without task-specific priors. To support systematic study, we propose GuessBench, a benchmark with both perception-oriented and knowledge-oriented images for evaluating active reasoning in MLLMs. We evaluate 20 superior MLLMs and find that performance on active reasoning lags far behind it on passive settings, indicating substantial room for improvement. Further analysis identifies fine-grained perception and timely decision-making as key challenges. Ablation studies show that perceptual enhancements benefit smaller models, whereas thinking-oriented methods provide consistent gains across model sizes. These results suggest promising directions for future research on multimodal active reasoning.

Related papers

• Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs [55.61018839017648]
  Chain-of-thought (CoT) reasoning has substantially improved multimodal large language models (MLLMs) on complex reasoning tasks.<n>Existing approaches largely rely on long textual reasoning trajectories and provide limited mechanisms for learning stable visual attention policies.<n>We propose SAYO, a visual reasoning model trained with a reinforcement learning framework that introduces a region-level visual attention-based reward.
  arXiv  Detail & Related papers  (2026-02-09T03:33:23Z)
• Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs [78.09559830840595]
  We present the first systematic study on quantizing diffusion-based language models.<n>We identify the presence of activation outliers, characterized by abnormally large activation values.<n>We implement state-of-the-art PTQ methods and conduct a comprehensive evaluation.
  arXiv  Detail & Related papers  (2025-08-20T17:59:51Z)
• From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information? [34.959850282872594]
  We present AR-Bench, a novel benchmark designed explicitly to evaluate an LLM's active reasoning skills.<n>AR-Bench comprises three task families-detective cases, situation puzzles, and guessing numbers.<n> Empirical evaluation on AR-Bench demonstrates that contemporary LLMs exhibit pronounced difficulties with active reasoning.
  arXiv  Detail & Related papers  (2025-06-09T23:56:41Z)
• Beyond Perception: Evaluating Abstract Visual Reasoning through Multi-Stage Task [22.16139464288789]
  Current Multimodal Large Language Models (MLLMs) excel in general visual reasoning but remain underexplored in abstract visual Reasoning (AVR)<n>Existing benchmarks focus on single-step reasoning, emphasizing the end result but neglecting the multi-stage nature of reasoning process.<n>We introduce MultiStAR, a Multi-Stage benchmark, designed to assess reasoning across varying levels of complexity.
  arXiv  Detail & Related papers  (2025-05-28T00:34:45Z)
• MLLMs are Deeply Affected by Modality Bias [158.64371871084478]
  Recent advances in Multimodal Large Language Models (MLLMs) have shown promising results in integrating diverse modalities such as texts and images.<n>MLLMs are heavily influenced by modality bias, often relying on language while under-utilizing other modalities like visual inputs.<n>This paper argues that MLLMs are deeply affected by modality bias, highlighting its manifestations across various tasks.
  arXiv  Detail & Related papers  (2025-05-24T11:49:31Z)
• Grounded Chain-of-Thought for Multimodal Large Language Models [66.04061083611863]
  We propose a new learning task for multimodal large language models (MLLMs) called Grounded Chain-of-Thought (GCoT)<n>GCoT is keen to helping MLLMs to recognize and ground the relevant visual cues step by step, thereby predicting the correct answer with grounding coordinates as the intuitive basis.<n>To facilitate this task, we also carefully design and construct a dataset called multimodal grounded chain-of-thought (MM-GCoT) consisting of 24,022 GCoT examples for 5,033 images.
  arXiv  Detail & Related papers  (2025-03-17T04:07:47Z)
• VOILA: Evaluation of MLLMs For Perceptual Understanding and Analogical Reasoning [63.0285363282581]
  Multimodal Large Language Models (MLLMs) have become a powerful tool for integrating visual and textual information.<n>We introduce VOILA, a benchmark designed to evaluate MLLMs' perceptual understanding and abstract relational reasoning.<n>We reveal that current MLLMs struggle to comprehend inter-image relationships and exhibit limited capabilities in high-level relational reasoning.
  arXiv  Detail & Related papers  (2025-02-25T23:36:19Z)
• ActiView: Evaluating Active Perception Ability for Multimodal Large Language Models [18.992215985625492]
  We evaluate active perception in Multimodal Large Language Models (MLLMs)<n>We focus on a specialized form of Visual Question Answering (VQA) that eases and quantifies the evaluation yet challenging for existing MLLMs.<n>We observe that restricted perceptual fields play a significant role in enabling active perception.
  arXiv  Detail & Related papers  (2024-10-07T00:16:26Z)
• Look Before You Decide: Prompting Active Deduction of MLLMs for Assumptive Reasoning [77.72128397088409]
  We show that most prevalent MLLMs can be easily fooled by the introduction of a presupposition into the question.<n>We also propose a novel reinforcement learning paradigm to encourage the model to actively perform composite deduction.
  arXiv  Detail & Related papers  (2024-04-19T15:53:27Z)
• Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection [34.135738700682055]
  This paper conducts a comprehensive evaluation of large language models (LLMs) represented by ChatGPT. We find that LLMs exhibit strong zero-shot and few-shot capabilities, but is still at a disadvantage compared to models fine-tuned with full resource.
  arXiv  Detail & Related papers  (2024-02-27T07:02:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.