Transparent Visual Reasoning via Object-Centric Agent Collaboration

• URL: http://arxiv.org/abs/2509.23757v1
• Date: Sun, 28 Sep 2025 09:06:52 GMT
• Title: Transparent Visual Reasoning via Object-Centric Agent Collaboration
• Abstract summary: Ocean is an inherently interpretable framework built on object-centric representations and a transparent multi-agent reasoning process.<n>We benchmark it against standard visual classifiers and popular posthoc explanation tools like GradCAM and LIME.<n>Our results demonstrate competitive performance with respect to state-of-the-art black-box models with a faithful reasoning process.
• Score: 24.503547825977023
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A central challenge in explainable AI, particularly in the visual domain, is producing explanations grounded in human-understandable concepts. To tackle this, we introduce OCEAN (Object-Centric Explananda via Agent Negotiation), a novel, inherently interpretable framework built on object-centric representations and a transparent multi-agent reasoning process. The game-theoretic reasoning process drives agents to agree on coherent and discriminative evidence, resulting in a faithful and interpretable decision-making process. We train OCEAN end-to-end and benchmark it against standard visual classifiers and popular posthoc explanation tools like GradCAM and LIME across two diagnostic multi-object datasets. Our results demonstrate competitive performance with respect to state-of-the-art black-box models with a faithful reasoning process, which was reflected by our user study, where participants consistently rated OCEAN's explanations as more intuitive and trustworthy.

Related papers

• Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning [82.86343313807158]
  We argue that a central challenge is how and when visual evidence is introduced into the reasoning process.<n>Motivated by this insight, we propose CSMR, a multimodal reasoning framework in which a language model controls the reasoning process.
  arXiv  Detail & Related papers  (2026-05-27T08:43:13Z)
• CAVE: A Structured Credit Assignment Approach for Fragmented Visual Evidence Reasoning [13.024838183478858]
  We propose Credit Assignment for Visual Evidence (CAVE) for interleaved visual reasoning.<n>CAVE evaluates the contribution of intermediate steps at the action level via three complementary reasoning process signals.<n>We construct TRACER-Bench, which covers four nonlocal and semantically confusable reasoning dimensions.
  arXiv  Detail & Related papers  (2026-05-13T16:50:24Z)
• Beyond Pixels: Visual Metaphor Transfer via Schema-Driven Agentic Reasoning [56.24016465596292]
  A visual metaphor constitutes a high-order form of human creativity, employing cross-domain semantic fusion to transform abstract concepts into impactful visual rhetoric.<n>We introduce the task of Visual Metaphor Transfer (VMT), which challenges models to autonomously decouple the "creative essence" from a reference image and re-materialize that abstract logic onto a user-specified subject.<n>Our method significantly outperforms SOTA baselines in metaphor consistency, analogy appropriateness, and visual creativity, paving the way for automated high-impact creative applications in advertising and media.
  arXiv  Detail & Related papers  (2026-02-01T17:01:36Z)
• Cognitive Inception: Agentic Reasoning against Visual Deceptions by Injecting Skepticism [81.39177645864757]
  We propose textbfInception, a fully reasoning-based agentic reasoning framework to conduct authenticity verification by injecting skepticism.<n>To the best of our knowledge, this is the first fully reasoning-based framework against AIGC visual deceptions.
  arXiv  Detail & Related papers  (2025-11-21T05:13:30Z)
• Explain Before You Answer: A Survey on Compositional Visual Reasoning [74.27548620675748]
  Compositional visual reasoning has emerged as a key research frontier in multimodal AI.<n>This survey systematically reviews 260+ papers from top venues (CVPR, ICCV, NeurIPS, ICML, ACL, etc.)<n>We then catalog 60+ benchmarks and corresponding metrics that probe compositional visual reasoning along dimensions such as grounding accuracy, chain-of-thought faithfulness, and high-resolution perception.
  arXiv  Detail & Related papers  (2025-08-24T11:01:51Z)
• A Cognitive Paradigm Approach to Probe the Perception-Reasoning Interface in VLMs [3.2228025627337864]
  This paper introduces a structured evaluation framework to dissect the perception-reasoning interface in Vision-Language Models (VLMs)<n>We propose three distinct evaluation paradigms, mirroring human problem-solving strategies.<n>Applying this framework, we demonstrate that CA, leveraging powerful language models for reasoning over rich, independently generated descriptions, achieves new state-of-the-art (SOTA) performance.
  arXiv  Detail & Related papers  (2025-01-23T12:42:42Z)
• Take A Step Back: Rethinking the Two Stages in Visual Reasoning [57.16394309170051]
  This paper revisits visual reasoning with a two-stage perspective. It is more efficient to implement symbolization via separated encoders for different data domains while using a shared reasoner. The proposed two-stage framework achieves impressive generalization ability on various visual reasoning tasks.
  arXiv  Detail & Related papers  (2024-07-29T02:56:19Z)
• Cantor: Inspiring Multimodal Chain-of-Thought of MLLM [83.6663322930814]
  We argue that converging visual context acquisition and logical reasoning is pivotal for tackling visual reasoning tasks. We propose an innovative multimodal CoT framework, termed Cantor, characterized by a perception-decision architecture. Our experiments demonstrate the efficacy of the proposed framework, showing significant improvements in multimodal CoT performance.
  arXiv  Detail & Related papers  (2024-04-24T17:59:48Z)
• Advancing Ante-Hoc Explainable Models through Generative Adversarial Networks [24.45212348373868]
  This paper presents a novel concept learning framework for enhancing model interpretability and performance in visual classification tasks. Our approach appends an unsupervised explanation generator to the primary classifier network and makes use of adversarial training. This work presents a significant step towards building inherently interpretable deep vision models with task-aligned concept representations.
  arXiv  Detail & Related papers  (2024-01-09T16:16:16Z)
• PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoning [135.2892665079159]
  We introduce a new large-scale diagnostic visual reasoning dataset named PTR. PTR contains around 70k RGBD synthetic images with ground truth object and part level annotations. We examine several state-of-the-art visual reasoning models on this dataset and observe that they still make many surprising mistakes.
  arXiv  Detail & Related papers  (2021-12-09T18:59:34Z)
• Interpretable Visual Reasoning via Induced Symbolic Space [75.95241948390472]
  We study the problem of concept induction in visual reasoning, i.e., identifying concepts and their hierarchical relationships from question-answer pairs associated with images. We first design a new framework named object-centric compositional attention model (OCCAM) to perform the visual reasoning task with object-level visual features. We then come up with a method to induce concepts of objects and relations using clues from the attention patterns between objects' visual features and question words.
  arXiv  Detail & Related papers  (2020-11-23T18:21:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.