Seeing Before Agreeing: Aligning Multi-Agent Consensus with Visual Evidence

• URL: http://arxiv.org/abs/2605.30698v1
• Date: Fri, 29 May 2026 00:45:25 GMT
• Title: Seeing Before Agreeing: Aligning Multi-Agent Consensus with Visual Evidence
• Abstract summary: Vision-language models (VLMs) have achieved strong performance on visual question answering (VQA)<n>To mitigate individual hallucinations and blind spots, aggregating diverse perspectives via multi-agent collaboration has emerged as a promising paradigm.<n>We propose a training-free evidence-centered framework for coordinating multiple VLM agents.
• Score: 13.363363690089407
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language models (VLMs) have achieved strong performance on visual question answering (VQA). To mitigate individual hallucinations and blind spots, aggregating diverse perspectives via multi-agent collaboration has emerged as a promising paradigm. While this approach has shown great success in textual QA, its potential in the multimodal domain remains under-explored. Existing multi-agent VQA methods predominantly adapt text-centric protocols, focusing on textual discussions while ignoring the alignment of visual information. In this work, we reveal a key insight: answer-level agreement is insufficient for reliable multi-agent VQA; \textit{aligned visual evidence} -- shared support from the image regions agents rely on -- is essential for trustworthy consensus. To leverage this insight, we propose EAGLE (\textbf{E}vidence-\textbf{A}ligned \textbf{G}rounded mu\textbf{L}ti-agent r\textbf{E}asoning), a training-free evidence-centered framework for coordinating multiple VLM agents. EAGLE explicitly exposes each agent's grounding regions as visual evidence, enables mutual verification over the evidence, and uses evidence consistency to guide final decision-making. Experiments on six VQA benchmarks show that EAGLE achieves best average performance across domains while remaining lightweight, interpretable, and practical for deployment.

Related papers

• UniRef-UAV: A Multimodal Benchmark for Universal Referring in UAV Imagery [69.87589293875112]
  Unmanned aerial vehicles (UAVs) increasingly rely on visual grounding capabilities to localize task-relevant targets.<n>Existing referring expression comprehension benchmarks and methods are largely built around text-only queries and single-object outputs.<n>We introduce emphUniversal Referring, a generalized UAV referring task that jointly expands the query modality and the output cardinality.
  arXiv  Detail & Related papers  (2026-07-09T09:09:31Z)
• VILLAIN at AVerImaTeC: Verifying Image-Text Claims via Multi-Agent Collaboration [10.712719361607753]
  VILLAIN is a multimodal fact-checking system that verifies image-text claims.<n>Our system ranked first on the leaderboard across all evaluation metrics.
  arXiv  Detail & Related papers  (2026-02-04T14:12:55Z)
• Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understanding [49.26132236798123]
  Vision Language Models (VLMs) have gradually become a primary approach in document understanding.<n>We propose SLEUTH, a multi agent framework that orchestrates a retriever and four collaborative agents in a coarse to fine process.<n>The framework identifies key textual and visual clues within the retrieved pages, filters for salient visual evidence such as tables and charts, and analyzes the query to devise a reasoning strategy.
  arXiv  Detail & Related papers  (2025-11-28T03:09:40Z)
• ReAG: Reasoning-Augmented Generation for Knowledge-based Visual Question Answering [54.72902502486611]
  ReAG is a Reasoning-Augmented Multimodal RAG approach that combines coarse- and fine-grained retrieval with a critic model that filters irrelevant passages.<n>ReAG significantly outperforms prior methods, improving answer accuracy and providing interpretable reasoning grounded in retrieved evidence.
  arXiv  Detail & Related papers  (2025-11-27T19:01:02Z)
• ProtoVQA: An Adaptable Prototypical Framework for Explainable Fine-Grained Visual Question Answering [30.65960340061612]
  Prototype-based modeling has shown promise for interpretability by grounding predictions in semantically meaningful regions.<n>We present ProtoVQA, a unified framework that learns question-aware prototypes that serve as reasoning anchors.<n>We propose the Visual-Linguistic Alignment Score (VLAS), which measures how well the model's attended regions align with ground-truth evidence.
  arXiv  Detail & Related papers  (2025-09-20T13:12:08Z)
• Visual Document Understanding and Question Answering: A Multi-Agent Collaboration Framework with Test-Time Scaling [83.78874399606379]
  We propose MACT, a Multi-Agent Collaboration framework with Test-Time scaling.<n>It comprises four distinct small-scale agents, with clearly defined roles and effective collaboration.<n>It shows superior performance with a smaller parameter scale without sacrificing the ability of general and mathematical tasks.
  arXiv  Detail & Related papers  (2025-08-05T12:52:09Z)
• Describe Anything Model for Visual Question Answering on Text-rich Images [7.618388911738171]
  We introduce DAM-QA, a framework to harness the region-aware capabilities from DAM for the text-rich Visual Question Answering problem.<n>Our approach consistently outperforms the baseline DAM, with a notable 7+ point gain on DocVQA.<n>Results highlight the potential of DAM-like models for text-rich and broader VQA tasks when paired with efficient usage and integration strategies.
  arXiv  Detail & Related papers  (2025-07-16T17:28:19Z)
• Benchmarking Retrieval-Augmented Multimodal Generation for Document Question Answering [60.062194349648195]
  Document Visual Question Answering (DocVQA) faces dual challenges in processing lengthy multimodal documents.<n>Current document retrieval-augmented generation (DocRAG) methods remain limited by their text-centric approaches.<n>We introduce MMDocRAG, a comprehensive benchmark featuring 4,055 expert-annotated QA pairs with multi-page, cross-modal evidence chains.
  arXiv  Detail & Related papers  (2025-05-22T09:52:57Z)
• VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation [100.06122876025063]
  This paper introduces VisDoMBench, the first comprehensive benchmark designed to evaluate QA systems in multi-document settings.<n>We propose VisDoMRAG, a novel multimodal Retrieval Augmented Generation (RAG) approach that simultaneously utilizes visual and textual RAG.
  arXiv  Detail & Related papers  (2024-12-14T06:24:55Z)
• Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answering [47.668572102657684]
  This work introduces a novel, explainable multi-agent collaboration framework by leveraging the expansive knowledge of Large Language Models (LLMs) to enhance the capabilities of Vision Language Models (VLMs)
  arXiv  Detail & Related papers  (2023-11-29T03:10:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.