GEAR-Seg: A Grounded Explainable Agent for Reasoning Segmentation and Data Engine

• URL: http://arxiv.org/abs/2607.00544v1
• Date: Wed, 01 Jul 2026 07:35:12 GMT
• Title: GEAR-Seg: A Grounded Explainable Agent for Reasoning Segmentation and Data Engine
• Abstract summary: GEAR-Seg transforms implicit reasoning into an explicit, trackable logic chain.<n>GEAR-Seg inherently functions as a highly scalable data engine.<n>Models supervised exclusively by our automated pipeline closely match the upper-bound performance of costly human-annotated baselines.
• Score: 8.312065094839399
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reasoning segmentation requires localizing targets based on complex, implicit queries. Current end-to-end models typically entangle perception and deduction into an opaque black box, severely limiting interpretability and scalability. To address this, we propose GEAR-Seg (Grounded Explainable Agent for Reasoning Segmentation), an explicitly decoupled agent that shifts the paradigm by translating visual pixels into dense, attribute-rich text. By decoupling class-agnostic segmentation, semantic description, and Large Language Model (LLM) deduction, GEAR-Seg transforms implicit reasoning into an explicit, trackable logic chain. As a zero-shot inference framework, it achieves highly competitive performance across diverse reasoning and fine-grained referring segmentation benchmarks. Furthermore, GEAR-Seg inherently functions as a highly scalable data engine. Utilizing this engine, we construct GEAR-131K, a massive benchmark (over 38k images, 656k QA-mask pairs) introducing a multifaceted taxonomy tailored for complex real-world manipulation-oriented reasoning. Finally, distillation experiments demonstrate that lightweight models supervised exclusively by our automated pipeline closely match the upper-bound performance of costly human-annotated baselines.

Related papers

• PiPS: Post-Hoc Prototypical Explanations for Interpretable Semantic Segmentation [1.4649457503983967]
  We propose PiPS (Post-hoc interpretable Prototypical), the first fully post-hoc solution for generating explanations for semantic segmentation models.<n>Our method enables the extraction of intuitive, spatially localized explanations from any pre-trained network without modification or fine-tuning.<n>This approach opens a new avenue for the safe and cost-effective deployment of transparent systems in advanced computer vision tasks.
  arXiv  Detail & Related papers  (2026-09-15T09:40:37Z)
• DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentation [5.5825182710791275]
  Reasoning segmentation aims to predict pixel-wise masks for targets given complex language queries.<n>We propose DGSeg, a framework that learns to fuse predictions guided by semantic and spatial cues.<n>Experiments demonstrate that DGSeg consistently outperforms strong baselines on multiple benchmarks.
  arXiv  Detail & Related papers  (2026-07-06T08:10:03Z)
• CR-Seg: Attention-Guided and CoT-Enhanced Coarse-to-Refined Reasoning Segmentation [26.21876180083083]
  Reasoning segmentation aims to segment target objects described by complex language through joint visual-textual reasoning.<n>Existing methods rely on either learned semantic tokens to bridge Multimodal Large Language Models (MLLMs) and segmentation models, suffering from difficult cross-modal alignment.<n>We propose Attention-Guided and CoT-Enhanced Coarse-to-Refined Reasoning, termed CR-Seg, a two-stage framework for coarse-to-refined spatial segmentation.
  arXiv  Detail & Related papers  (2026-06-02T12:30:04Z)
• Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction [127.64173950476702]
  We study direct corpus interaction (DCI), where an agent searches the raw corpus directly with general-purpose terminal tools.<n>This approach requires no offline indexing and adapts naturally to evolving local corpora.<n>Across IR benchmarks and end-to-end agentic search tasks, this simple setup substantially outperforms strong sparse, dense, and reranking baselines.
  arXiv  Detail & Related papers  (2026-05-03T19:13:11Z)
• Spatial Atlas: Compute-Grounded Reasoning for Spatial-Aware Research Agent Benchmarks [1.5229257192293202]
  We introduce compute-grounded reasoning (CGR) for spatial-aware research agents.<n>CGR is a design paradigm for spatial-aware research agents in which every answerable sub-problem is resolved by deterministic computation.<n>We show that CGR yields competitive accuracy while maintaining interpretability.
  arXiv  Detail & Related papers  (2026-04-13T22:22:07Z)
• CORE-Seg: Reasoning-Driven Segmentation for Complex Lesions via Reinforcement Learning [48.77689764063103]
  We introduce ComLesion-14K, the first diverse Chain-of-Thought benchmark for reasoning-driven complex lesion segmentation.<n>We propose CORE-Seg, an end-to-end framework integrating reasoning with segmentation through a Semantic-Guided Prompt Adapter.<n>Our Method achieves state-of-the-art results with a mean Dice of 37.06% (14.89% higher than the second-best baseline), while reducing the failure rate to 18.42%.
  arXiv  Detail & Related papers  (2026-03-06T04:58:55Z)
• CoT-Seg: Rethinking Segmentation with Chain-of-Thought Reasoning and Self-Correction [50.67483317563736]
  This paper aims to explore a system that can think step-by-step, look up information if needed, generate results, self-evaluate its own results, and refine the results.<n>We introduce CoT-Seg, a training-free framework that rethinks reasoning segmentation by combining chain-of-thought reasoning with self-correction.
  arXiv  Detail & Related papers  (2026-01-24T11:41:54Z)
• A Reasoning Paradigm for Named Entity Recognition [16.86833034216367]
  Reasoning framework is proposed for Named Entity Recognition.<n> framework consists of three stages: Chain of Thought (CoT) generation, CoT tuning, and reasoning enhancement.<n>Experiments show ReasoningNER demonstrates impressive cognitive ability in the NER task, achieving competitive performance.
  arXiv  Detail & Related papers  (2025-11-15T01:31:43Z)
• LENS: Learning to Segment Anything with Unified Reinforced Reasoning [38.582392908238866]
  We introduce LENS, a scalable reinforcement-learning framework that jointly optimize the reasoning process and segmentation in an end-to-end manner.<n>LENS achieves an average cIoU of 81.2% on the RefCOCO, RefCOCO+, and RefCOCOg benchmarks, outperforming the strong fine-tuned method, i.e., GLaMM, by up to 5.6%.
  arXiv  Detail & Related papers  (2025-08-19T17:59:53Z)
• Think Before You Segment: High-Quality Reasoning Segmentation with GPT Chain of Thoughts [64.93416171745693]
  ThinkFirst is a training-free reasoning segmentation framework.<n>Our approach allows GPT-4o or other powerful MLLMs to generate a detailed, chain-of-thought description of an image.<n>This summarized description is then passed to a language-instructed segmentation assistant to aid the segmentation process.
  arXiv  Detail & Related papers  (2025-03-10T16:26:11Z)
• LISA: Reasoning Segmentation via Large Language Model [68.24075852136761]
  We propose a new segmentation task -- reasoning segmentation. The task is designed to output a segmentation mask given a complex and implicit query text. We present LISA: large Language Instructed Assistant, which inherits the language generation capabilities of multimodal Large Language Models.
  arXiv  Detail & Related papers  (2023-08-01T17:50:17Z)
• Context-aware Fine-tuning of Self-supervised Speech Models [56.95389222319555]
  We study the use of context, i.e., surrounding segments, during fine-tuning. We propose a new approach called context-aware fine-tuning. We evaluate the proposed approach using the SLUE and Libri-light benchmarks for several downstream tasks.
  arXiv  Detail & Related papers  (2022-12-16T15:46:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.