UnAC: Adaptive Visual Prompting with Abstraction and Stepwise Checking for Complex Multimodal Reasoning

• URL: http://arxiv.org/abs/2605.03950v1
• Date: Tue, 05 May 2026 16:36:58 GMT
• Title: UnAC: Adaptive Visual Prompting with Abstraction and Stepwise Checking for Complex Multimodal Reasoning
• Abstract summary: UnAC is a multimodal prompting method that strengthens reasoning for complex multimodal tasks.<n>We propose an adaptive visual prompting strategy that enables LMMs to focus on salient regions.<n>We also introduce a gradual self-checking scheme that improves reasoning by verifying each subquestion and its answer.
• Score: 36.66655623392861
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although recent LMMs have become much stronger at visual perception, they remain unreliable on problems that require multi-step reasoning over visual evidence. In this paper, we present UnAC (Understanding, Abstracting, and Checking), a multimodal prompting method that strengthens reasoning for complex multimodal tasks in LMMs (e.g., GPT-4o, Gemini 1.5, and GPT-4V). To improve image understanding and capture fine details, we propose an adaptive visual prompting strategy that enables LMMs to focus on salient regions. We further design an image-abstraction prompt to effectively extract key information from images. In addition, we introduce a gradual self-checking scheme that improves reasoning by verifying each decomposed subquestion and its answer. Extensive experiments on three public benchmarks-MathVista, MM-Vet, and MMMU.

Related papers

• QG-CoC: Question-Guided Chain-of-Captions for Large Multimodal Models [50.51641024244313]
  We investigate how current prompting methods perceive fine-grained visual details and process visual information when dealing with multiple images.<n>Inspired by the findings, we propose a new zero-shot prompting method, Question-Guided Chain-of-Captions (QG-CoC)<n>We evaluate our method on various open-source and closed-source MLLMs for multi-image and single-image benchmarks.
  arXiv  Detail & Related papers  (2025-11-05T05:49:48Z)
• Directed-Tokens: A Robust Multi-Modality Alignment Approach to Large Language-Vision Models [28.82265769298008]
  We introduce a simple but efficient learning mechanism for improving the robust alignment between visual and textual modalities.<n>The proposed approach consistently achieves state-of-the-art (SoTA) performance compared with prior LMMs.
  arXiv  Detail & Related papers  (2025-08-19T20:53:24Z)
• Meta-Adaptive Prompt Distillation for Few-Shot Visual Question Answering [53.82094608038132]
  Large Multimodal Models (LMMs) often rely on in-context learning (ICL) to perform new tasks with minimal supervision.<n>ICL performance, especially in smaller LMMs, is inconsistent and does not always improve monotonically with increasing examples.<n>We propose a meta-learning approach that provides an alternative for inducing few-shot capabilities in LMMs.
  arXiv  Detail & Related papers  (2025-06-07T19:37:22Z)
• Mitigating Visual Forgetting via Take-along Visual Conditioning for Multi-modal Long CoT Reasoning [53.790502697674754]
  We propose Take-along Visual Conditioning (TVC), a strategy that shifts image input to critical reasoning stages.<n>TVC helps the model retain attention to the visual components throughout the reasoning.<n>Our approach achieves state-of-the-art performance on average across five mathematical reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-03-17T16:45:12Z)
• Visualizing Thought: Conceptual Diagrams Enable Robust Planning in LMMs [59.66595230543127]
  Conceptual diagrams externalize mental models, abstracting irrelevant details to efficiently capture how entities interact.<n>Large Language Models (LLMs) and Large MultiModal Models (LMMs) predominantly reason through text.<n>We propose Visual Thinking, a generalizable framework that enables LMMs to reason through multiple chains of self-generated conceptual diagrams.
  arXiv  Detail & Related papers  (2025-03-14T18:27:02Z)
• ToFu: Visual Tokens Reduction via Fusion for Multi-modal, Multi-patch, Multi-image Task [34.269081635534526]
  We propose ToFu, a visual encoder-agnostic, training-free Token Fusion strategy for high-resolution, multi-image, tasks.<n>We validate our approach on the well-established LLaVA-Interleave Bench, which covers challenging multi-image tasks.
  arXiv  Detail & Related papers  (2025-03-06T14:00:59Z)
• VOILA: Evaluation of MLLMs For Perceptual Understanding and Analogical Reasoning [63.0285363282581]
  Multimodal Large Language Models (MLLMs) have become a powerful tool for integrating visual and textual information.<n>We introduce VOILA, a benchmark designed to evaluate MLLMs' perceptual understanding and abstract relational reasoning.<n>We reveal that current MLLMs struggle to comprehend inter-image relationships and exhibit limited capabilities in high-level relational reasoning.
  arXiv  Detail & Related papers  (2025-02-25T23:36:19Z)
• Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Models [14.765057045747753]
  Chain-of-Thought (CoT) and related rationale-based works have significantly improved the performance of Large Language Models (LLMs) in complex reasoning tasks. We propose the Image-of-Thought (IoT) prompting method, which helps MLLMs to extract visual rationales step-by-step. IoT prompting has improved zero-shot visual reasoning performance across various visual understanding tasks in different MLLMs.
  arXiv  Detail & Related papers  (2024-05-22T17:56:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.