SeGDeP: Semantic- and Geometric-Aware Decoupled Prompts for Reasoning Segmentation

• URL: http://arxiv.org/abs/2609.08867v1
• Date: Tue, 08 Sep 2026 15:11:32 GMT
• Title: SeGDeP: Semantic- and Geometric-Aware Decoupled Prompts for Reasoning Segmentation
• Abstract summary: Reasoning segmentation converts an implicit linguistic conclusion into a precise mask.<n>Existing MLLM-segmenter interfaces either use a special trigger or compress both signals into one context.<n>We present SeGDeP, an explicit what-where interface.
• Score: 51.46828526392219
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reasoning segmentation converts an implicit linguistic conclusion into a precise mask, requiring both semantic identification and spatial grounding. Existing MLLM-segmenter interfaces either use a special trigger or compress both signals into one context, although they receive different supervision and fail differently. This coupling obscures whether a failure arises from target interpretation or from localization. We present SeGDeP, an explicit what-where interface. A semantic prompt branch and an independent geometric projection path transform resolved MLLM states into semantic features and a DETR-predicted box, which jointly condition a SAM 3 mask decoder. Training first aligns this executable interface, then uses group reward-decoupled policy optimization (GDPO) to balance format, box-IoU, and mask-IoU feedback. SeGDeP-4B reaches 82.7 average cIoU over eight RefCOCO-family splits and 66.0/59.6 gIoU on ReasonSeg val/test while adapting only 0.38% of Qwen3-VL parameters through LoRA. Controlled stage-wise ablations, gradient diagnostics, and prompt interventions further show that the two paths develop complementary semantic and geometric specialization rather than duplicating the same evidence.

Related papers

• A*-Thought-V2: Efficient Latent Reasoning via Geometric Dynamics of LLM [40.07171947823021]
  Chain-of-Thought (CoT) improves the reasoning ability of Large Language Models (LLMs) but incurs substantial computation and context costs.<n>We present A*-Thought-V2, a geometric dynamics of LLM guided framework that models CoT as a hidden-state trajectory and replaces hard deletion with an explicit-implicit interleaved latent architecture.
  arXiv  Detail & Related papers  (2026-09-07T17:56:20Z)
• Learning Where Outcomes Change:Credit-Addressable Reasoning for Multimodal Geometry [105.89493240787861]
  We introduce credit-addressable reasoning, in which semantic units exposed during inference also define where learning compares alternatives and assigns credit.<n>We instantiate this principle with Code-CoT, which represents visual relations as line-addressable executable code, and organizes reasoning into typed events.<n>Across nine geometry benchmarks, CE-GRPO achieves an average accuracy of 76.04, outperforming Qwen3-VL-8B and trajectory-level GRPO by $8.09$ and 3.43 points, respectively.
  arXiv  Detail & Related papers  (2026-08-31T08:44:36Z)
• DRAgent: Discriminative Reasoning Agent for Referring Expression Segmentation [0.0]
  DRAgent is an MLLM-driven discriminative reasoning framework for Referring Expression.<n>DRAgent achieves competitive performance on RefCOCO, RefCOCO+, and RefCOCOg.
  arXiv  Detail & Related papers  (2026-08-24T07:18:38Z)
• Steering Vision-Language Models with Joint Sparse Autoencoders [2.367986397868328]
  We introduce the Joint Sparse Autoencoder (JSAE) to factorize sequence-pooled vision and language activations into shared, interpretable image/caption-level features.<n>Applying JSAE to vision-language models (VLMs) yields representations that are difficult to use as controllable cross-modal steering directions.
  arXiv  Detail & Related papers  (2026-06-24T10:11:04Z)
• SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning [99.49739831030388]
  We study how the design of this interface shapes the agent's capacity for open-ended spatial reasoning.<n>We propose SpatialClaw, a training-free framework for spatial reasoning that adopts code as the action interface.
  arXiv  Detail & Related papers  (2026-06-11T17:59:36Z)
• SegRAG: Training-Free Retrieval-Augmented Semantic Segmentation [13.665861251747144]
  SegRAG is a training-free retrieval-augmented segmentation framework.<n>It grounds SAM3 with class-specific point prompts derived from a curated DINOv3 feature bank.<n>On four standard benchmarks, SegRAG consistently outperforms the text-only baseline.
  arXiv  Detail & Related papers  (2026-05-17T19:51:32Z)
• Trans-RAG: Query-Centric Vector Transformation for Secure Cross-Organizational Retrieval [16.56643936191435]
  Trans-RAG implements a novel vector space language paradigm where each organization's knowledge exists in a mathematically isolated semantic space.<n>Tests show minimal accuracy degradation (3.5% decrease in nDCG@10) and significant efficiency improvements over homomorphic encryption.
  arXiv  Detail & Related papers  (2026-04-10T17:58:06Z)
• RSRefSeg 2: Decoupling Referring Remote Sensing Image Segmentation with Foundation Models [25.265113510539546]
  Referring Remote Sensing Image provides a flexible and fine-grained framework for remote sensing scene analysis.<n>Current approaches utilize a three-stage pipeline encompassing dual-modal encoding, cross-modal interaction, and pixel decoding.<n>We propose RSRefSeg 2, a decoupling paradigm that reformulates the conventional workflow into a collaborative dual-stage framework.
  arXiv  Detail & Related papers  (2025-07-08T17:59:58Z)
• Hierarchical Context Alignment with Disentangled Geometric and Temporal Modeling for Semantic Occupancy Prediction [61.484280369655536]
  Camera-based 3D Semantic Occupancy Prediction (SOP) is crucial for understanding complex 3D scenes from limited 2D image observations.<n>Existing SOP methods typically aggregate contextual features to assist the occupancy representation learning.<n>We introduce a new Hierarchical context alignment paradigm for a more accurate SOP (Hi-SOP)
  arXiv  Detail & Related papers  (2024-12-11T09:53:10Z)
• LESS: Label-Efficient and Single-Stage Referring 3D Segmentation [55.06002976797879]
  Referring 3D is a visual-language task that segments all points of the specified object from a 3D point cloud described by a sentence of query. We propose a novel Referring 3D pipeline, Label-Efficient and Single-Stage, dubbed LESS, which is only under the supervision of efficient binary mask. We achieve state-of-the-art performance on ScanRefer dataset by surpassing the previous methods about 3.7% mIoU using only binary labels.
  arXiv  Detail & Related papers  (2024-10-17T07:47:41Z)
• HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learning [74.76431541169342]
  Zero-shot learning (ZSL) tackles the unseen class recognition problem, transferring semantic knowledge from seen classes to unseen ones. We propose a novel hierarchical semantic-visual adaptation (HSVA) framework to align semantic and visual domains. Experiments on four benchmark datasets demonstrate HSVA achieves superior performance on both conventional and generalized ZSL.
  arXiv  Detail & Related papers  (2021-09-30T14:27:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.