Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning

• URL: http://arxiv.org/abs/2411.18203v5
• Date: Wed, 23 Apr 2025 17:46:47 GMT
• Title: Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning
• Authors: Di Zhang, Junxian Li, Jingdi Lei, Xunzhi Wang, Yujie Liu, Zonglin Yang, Jiatong Li, Weida Wang, Suorong Yang, Jianbo Wu, Peng Ye, Wanli Ouyang, Dongzhan Zhou,
• Abstract summary: Critic-V is a framework inspired by the Actor-Critic paradigm to boost the reasoning capability of vision-language models (VLMs)<n>The Reasoner generates reasoning paths based on visual and textual inputs, and the Critic provides constructive critique to refine these paths.<n> evaluation results show that the Critic-V framework significantly outperforms existing methods, including GPT-4V, on 5 out of 8 benchmarks.
• Score: 46.411313304605564
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language models (VLMs) have shown remarkable advancements in multimodal reasoning tasks. However, they still often generate inaccurate or irrelevant responses due to issues like hallucinated image understandings or unrefined reasoning paths. To address these challenges, we introduce Critic-V, a novel framework inspired by the Actor-Critic paradigm to boost the reasoning capability of VLMs. This framework decouples the reasoning process and critic process by integrating two independent components: the Reasoner, which generates reasoning paths based on visual and textual inputs, and the Critic, which provides constructive critique to refine these paths. In this approach, the Reasoner generates reasoning responses according to text prompts, which can evolve iteratively as a policy based on feedback from the Critic. This interaction process was theoretically driven by a reinforcement learning framework where the Critic offers natural language critiques instead of scalar rewards, enabling more nuanced feedback to boost the Reasoner's capability on complex reasoning tasks. The Critic model is trained using Direct Preference Optimization (DPO), leveraging a preference dataset of critiques ranked by Rule-based Reward~(RBR) to enhance its critic capabilities. Evaluation results show that the Critic-V framework significantly outperforms existing methods, including GPT-4V, on 5 out of 8 benchmarks, especially regarding reasoning accuracy and efficiency. Combining a dynamic text-based policy for the Reasoner and constructive feedback from the preference-optimized Critic enables a more reliable and context-sensitive multimodal reasoning process. Our approach provides a promising solution to enhance the reliability of VLMs, improving their performance in real-world reasoning-heavy multimodal applications such as autonomous driving and embodied intelligence.

Related papers

• The Lighthouse of Language: Enhancing LLM Agents via Critique-Guided Improvement [49.687224320842105]
  Large language models (LLMs) have recently transformed from text-based assistants to autonomous agents capable of planning, reasoning, and iteratively improving their actions. In this work, we introduce Critique-Guided Improvement (CGI), a novel two-player framework, comprising an actor model that explores an environment and a critic model that generates detailed nature language feedback.
  arXiv  Detail & Related papers  (2025-03-20T10:42:33Z)
• RealCritic: Towards Effectiveness-Driven Evaluation of Language Model Critiques [59.861013614500024]
  We introduce a new benchmark designed to assess the critique capabilities of Large Language Models (LLMs) Unlike existing benchmarks, which typically function in an open-loop fashion, our approach employs a closed-loop methodology that evaluates the quality of corrections generated from critiques.
  arXiv  Detail & Related papers  (2025-01-24T13:48:10Z)
• VISCO: Benchmarking Fine-Grained Critique and Correction Towards Self-Improvement in Visual Reasoning [112.35483894933904]
  We propose VISCO, the first benchmark to extensively analyze the fine-grained critique and correction capabilities of LVLMs. VISCO features dense and fine-grained critique, requiring LVLMs to evaluate the correctness of each step in the chain-of-thought. LookBack significantly improves critique and correction performance by up to 13.5%.
  arXiv  Detail & Related papers  (2024-12-03T05:04:49Z)
• Critic-CoT: Boosting the reasoning abilities of large language model via Chain-of-thoughts Critic [48.94340387130627]
  Critic-CoT is a framework that pushes LLMs toward System-2-like critic capability. CoT reasoning paradigm and the automatic construction of distant-supervision data without human annotation. Experiments on GSM8K and MATH demonstrate that our enhanced model significantly boosts task-solving performance.
  arXiv  Detail & Related papers  (2024-08-29T08:02:09Z)
• CriticBench: Benchmarking LLMs for Critique-Correct Reasoning [26.45110574463893]
  CriticBench is a benchmark designed to assess Large Language Models' abilities to critique and rectify their reasoning. We evaluate and dissect the performance of 17 LLMs in generation, critique, and correction reasoning.
  arXiv  Detail & Related papers  (2024-02-22T18:59:02Z)
• CriticEval: Evaluating Large Language Model as Critic [110.29766259843453]
  CriticEval is a novel benchmark designed to comprehensively and reliably evaluate critique ability of Large Language Models. To ensure the comprehensiveness, CriticEval evaluates critique ability from four dimensions across nine diverse task scenarios. To ensure the reliability, a large number of critiques are annotated to serve as references.
  arXiv  Detail & Related papers  (2024-02-21T12:38:59Z)
• REFINER: Reasoning Feedback on Intermediate Representations [47.36251998678097]
  We introduce REFINER, a framework for finetuning language models to generate intermediate inferences. REFINER works by interacting with a critic model that provides automated feedback on the reasoning. Empirical evaluations show significant improvements over baseline LMs of comparable scale.
  arXiv  Detail & Related papers  (2023-04-04T15:57:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.