PixelEyes: Decoupling Perception and Reasoning for Pinpoint Visual Evidence Seeking

• URL: http://arxiv.org/abs/2607.00115v1
• Date: Tue, 30 Jun 2026 19:51:54 GMT
• Title: PixelEyes: Decoupling Perception and Reasoning for Pinpoint Visual Evidence Seeking
• Abstract summary: This paper explores multi-turn visual reasoning and observes that MLLMs repeatedly fail to localize the target.<n>We propose PixelEyes, a visual reasoning agent that explicitly decouples reasoning from perception.
• Score: 63.574885296709716
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper explores multi-turn visual reasoning and observes that MLLMs repeatedly fail to localize the target, leading to long, redundant trajectories. We attribute this failure to the entanglement of reasoning and perception within a single model, the MLLM reasons and localizes simultaneously, and inaccurate localization triggers additional reasoning turns that bloat the trajectory. To solve this problem, we propose PixelEyes, a multi-turn visual reasoning agent that explicitly decouples reasoning from perception, i.e., the reasoner decides what to look for, while a specialized perception tool answers where it is. Specifically, PixelEyes introduces 1) Mask-guided Visual Search. A referring segmentation model is invoked to provide mask-precise localization, freeing the reasoner from the need to compensate for imprecise grounding. 2) Semantic-region Breadth-first Search (BFS). To eliminate redundant loops caused by repeatedly cropping incorrect sub-regions, we organize exploration as a breadth-first search over semantic regions. To internalize these capabilities, we construct the PixelEyes-6K dataset by resynthesizing expert trajectories from existing data. This explicitly embeds our mask-guided search and BFS logic into the model. We further introduce Pinpoint-Bench, a zero-hint visual search benchmark, i.e., no location cues are provided in the question, with instance-level masks and bounding boxes that separate localization failures from reasoning failures, enabling fine-grained analysis of failure modes such as inattentional blindness. Recent state-of-the-art MLLMs and visual reasoning agents leave large headroom on Pinpoint-Bench, demonstrating its quality and difficulty. Code and models are open-sourced.

Related papers

• LARAD: Layout-Aware Road Anomaly Detection via Spatial-Logic Reasoning [16.92190487823839]
  We propose layout-Aware Road Anomaly Detection (LARAD), which shifts the paradigm from appearance matching to spatial-logic reasoning.<n>First, we introduce the Spatial-Logic Violation Synthesis pipeline, which generates training samples that are texture-consistent yet spatially invalid.<n>Second, we augment a standard closed-set segmentation network with a lightweight, OoD-guided attention branch.
  arXiv  Detail & Related papers  (2026-07-14T15:09:00Z)
• LADBench: A Benchmark for Logical Fault Detection in Images [1.6882040908691864]
  Large Vision Language Models (VLMs) excel at visual question answering and semantic grounding, but their capacity for autonomous logical reasoning remains underexplored.<n>Existing anomaly benchmarks emphasize visual errors or direct prompting rather than the physical and social common sense needed for open-world deployment.<n>We introduce LAD-bench, a benchmark of more than 1,000 curated synthetic images with logical anomalies across four domains.
  arXiv  Detail & Related papers  (2026-06-16T02:32:38Z)
• Reasoning-Driven Anomaly Detection and Localization with Image-Level Supervision [41.94832792442927]
  In this work, we activate the intrinsic reasoning potential of MLLMs to perform anomaly detection, pixel-level localization, and interpretable reasoning solely from image-level supervision.<n>Specifically, we propose Reasoning-Driven Anomaly localization (ReAL), which extracts anomaly-related tokens from the autoregressive reasoning process and aggregates their attention responses to produce pixel-level anomaly maps.
  arXiv  Detail & Related papers  (2026-03-28T07:34:33Z)
• Context-Nav: Context-Driven Exploration and Viewpoint-Aware 3D Spatial Reasoning for Instance Navigation [4.039245878626346]
  Text-goal instance navigation (TGIN) asks an agent to resolve a single, free-form description into actions that reach the correct object instance.<n>We present textitContext-Nav, which elevates long, contextual captions from a local cue to a global exploration prior.
  arXiv  Detail & Related papers  (2026-03-10T11:08:35Z)
• Seeing Before Reasoning: A Unified Framework for Generalizable and Explainable Fake Image Detection [58.82268659497348]
  We argue that the root of this failure lies in a fundamental mismatch: MLLMs are asked to reason about fakes before they can truly see them.<n>We propose Forensic-Chat, a generalizable, explainable, and still-conversational assistant for fake image detection.
  arXiv  Detail & Related papers  (2025-09-29T20:59:19Z)
• ERGO: Efficient High-Resolution Visual Understanding for Vision-Language Models [11.263321053154364]
  ERGO is a reasoning-driven perception-leveraging multimodal context to determine where to focus.<n>We develop simple yet effective reward components in a reinforcement learning framework for coarse-to-fine perception.<n>Our approach delivers higher accuracy than the original model and competitive methods, with greater efficiency.
  arXiv  Detail & Related papers  (2025-09-26T07:15:19Z)
• Can MLLMs Guide Me Home? A Benchmark Study on Fine-Grained Visual Reasoning from Transit Maps [56.76175383189738]
  We introduce ReasonMap, a benchmark designed to assess the fine-grained visual understanding and spatial reasoning abilities of MLLMs.<n>ReasonMap encompasses high-resolution transit maps from 30 cities across 13 countries and includes 1,008 question-answer pairs spanning two question types and three templates.<n> Comprehensive evaluations of 15 popular MLLMs, including both base and reasoning variants, reveal a counterintuitive pattern.
  arXiv  Detail & Related papers  (2025-05-24T12:33:52Z)
• Are Multimodal Large Language Models Ready for Omnidirectional Spatial Reasoning? [66.88619941063048]
  We ask: Are multimodal large language models (MLLMs) ready for omnidirectional spatial reasoning?<n> OSR-Bench is the first benchmark specifically designed for this setting.<n>It includes over 153,000 diverse question-answer pairs grounded in high-fidelity panoramic indoor scene maps.<n>We evaluate eight state-of-the-art MLLMs, including GPT-4o, Gemini 1.5 Pro, and leading open-source models under zero-shot settings.
  arXiv  Detail & Related papers  (2025-05-17T08:48:40Z)
• Exposing Attention Glitches with Flip-Flop Language Modeling [55.0688535574859]
  This work identifies and analyzes the phenomenon of attention glitches in large language models. We introduce flip-flop language modeling (FFLM), a family of synthetic benchmarks designed to probe the extrapolative behavior of neural language models. We find that Transformer FFLMs suffer from a long tail of sporadic reasoning errors, some of which we can eliminate using various regularization techniques.
  arXiv  Detail & Related papers  (2023-06-01T17:44:35Z)
• Reference-based Defect Detection Network [57.89399576743665]
  The first issue is the texture shift which means a trained defect detector model will be easily affected by unseen texture. The second issue is partial visual confusion which indicates that a partial defect box is visually similar with a complete box. We propose a Reference-based Defect Detection Network (RDDN) to tackle these two problems.
  arXiv  Detail & Related papers  (2021-08-10T05:44:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.