AnchorSeg: Language Grounded Query Banks for Reasoning Segmentation

• URL: http://arxiv.org/abs/2604.18562v3
• Date: Wed, 22 Apr 2026 02:31:50 GMT
• Title: AnchorSeg: Language Grounded Query Banks for Reasoning Segmentation
• Abstract summary: We introduce AnchorSeg, which reformulates reasoning segmentation as a structured conditional generation process over image tokens.<n>We model spatial conditioning as a factorized distribution over image tokens, where the anchor query determines localization signals.<n>We propose Token--Mask Cycle Consistency (TMCC), a bidirectional training objective that enforces alignment across resolutions.
• Score: 56.21301367698041
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reasoning segmentation requires models to ground complex, implicit textual queries into precise pixel-level masks. Existing approaches rely on a single segmentation token $\texttt{<SEG>}$, whose hidden state implicitly encodes both semantic reasoning and spatial localization, limiting the model's ability to explicitly disentangle what to segment from where to segment. We introduce AnchorSeg, which reformulates reasoning segmentation as a structured conditional generation process over image tokens, conditioned on language grounded query banks. Instead of compressing all semantic reasoning and spatial localization into a single embedding, AnchorSeg constructs an ordered sequence of query banks: latent reasoning tokens that capture intermediate semantic states, and a segmentation anchor token that provides explicit spatial grounding. We model spatial conditioning as a factorized distribution over image tokens, where the anchor query determines localization signals while contextual queries provide semantic modulation. To bridge token-level predictions and pixel-level supervision, we propose Token--Mask Cycle Consistency (TMCC), a bidirectional training objective that enforces alignment across resolutions. By explicitly decoupling spatial grounding from semantic reasoning through structured language grounded query banks, AnchorSeg achieves state-of-the-art results on ReasonSeg test set (67.7\% gIoU and 68.1\% cIoU). All code and models are publicly available at https://github.com/rui-qian/AnchorSeg.

Related papers

• Better, Stronger, Faster, and Broader: Structured All-Mask Prediction for MLLM-Based Segmentation [10.813064862132379]
  STAMP combines strong referring and reasoning segmentation with preserved multimodal ability and efficient inference.<n>It generates a target list with explicit IDs and optional boxes, binds these IDs to a shared multi-class mask space, and jointly predicts all targets in one non-autoregressive pass.<n>A single unified checkpoint retains STAMP's referring and reasoning capabilities while extending to open-vocabulary semantic, instance-aware, and remote-sensing small-target segmentation.
  arXiv  Detail & Related papers  (2026-08-03T18:42:11Z)
• DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentation [5.5825182710791275]
  Reasoning segmentation aims to predict pixel-wise masks for targets given complex language queries.<n>We propose DGSeg, a framework that learns to fuse predictions guided by semantic and spatial cues.<n>Experiments demonstrate that DGSeg consistently outperforms strong baselines on multiple benchmarks.
  arXiv  Detail & Related papers  (2026-07-06T08:10:03Z)
• Progressive Prompt-Guided Cross-Modal Reasoning for Referring Image Segmentation [11.276795416626385]
  Referring image segmentation aims to localize and segment a target object in an image based on a free-form referring expression.<n>We propose PPCR, a Progressive Prompt-guided Cross-modal Reasoning framework referring image segmentation.<n> PPCR explicitly structures the reasoning process as a Semantic-Spatial Grounding-Instance pipeline.
  arXiv  Detail & Related papers  (2026-03-30T03:33:10Z)
• CORA: Consistency-Guided Semi-Supervised Framework for Reasoning Segmentation [54.53371540755023]
  Reasoning segmentation seeks pixel-accurate masks for targets referenced by complex, often implicit instructions.<n>We present CORA, a semi-supervised reasoning segmentation framework that jointly learns from limited labeled data and a large corpus of unlabeled images.<n> CORA achieves state-of-the-art results, requiring as few as 100 labeled images on Cityscapes, a benchmark dataset for urban scene understanding.
  arXiv  Detail & Related papers  (2025-11-21T20:14:55Z)
• SegEarth-R1: Geospatial Pixel Reasoning via Large Language Model [61.97017867656831]
  We introduce a new task, ie, geospatial pixel reasoning, which allows implicit querying and reasoning and generates the mask of the target region.<n>We construct and release the first large-scale benchmark dataset called EarthReason, which comprises 5,434 manually annotated image masks with over 30,000 implicit question-answer pairs.<n>SegEarth-R1 achieves state-of-the-art performance on both reasoning and referring segmentation tasks, significantly outperforming traditional and LLM-based segmentation methods.
  arXiv  Detail & Related papers  (2025-04-13T16:36:47Z)
• LESS: Label-Efficient and Single-Stage Referring 3D Segmentation [55.06002976797879]
  Referring 3D is a visual-language task that segments all points of the specified object from a 3D point cloud described by a sentence of query. We propose a novel Referring 3D pipeline, Label-Efficient and Single-Stage, dubbed LESS, which is only under the supervision of efficient binary mask. We achieve state-of-the-art performance on ScanRefer dataset by surpassing the previous methods about 3.7% mIoU using only binary labels.
  arXiv  Detail & Related papers  (2024-10-17T07:47:41Z)
• DQFormer: Towards Unified LiDAR Panoptic Segmentation with Decoupled Queries [14.435906383301555]
  We propose a novel framework dubbed DQFormer to implement semantic and instance segmentation in a unified workflow. Specifically, we design a decoupled query generator to propose informative queries with semantics by localizing things/stuff positions. We also introduce a query-oriented mask decoder to decode corresponding segmentation masks.
  arXiv  Detail & Related papers  (2024-08-28T14:14:33Z)
• Learning Spatial-Semantic Features for Robust Video Object Segmentation [108.045326229865]
  We propose a robust video object segmentation framework that learns spatial-semantic features and discriminative object queries.<n>The proposed method achieves state-of-the-art performance on benchmark data sets, including the DAVIS 2017 test (textbf87.8%), YoutubeVOS 2019 (textbf88.1%), MOSE val (textbf74.0%), and LVOS test (textbf73.0%)
  arXiv  Detail & Related papers  (2024-07-10T15:36:00Z)
• Tokenize Anything via Prompting [65.93061853439512]
  We present a unified, promptable model capable of simultaneously segmenting, recognizing, and captioning anything. We train a generalizable model with massive segmentation masks, eg, SA-1B masks, and semantic priors from a pre-trained CLIP model with 5 billion parameters. We believe this model can be a versatile region-level image tokenizer, capable of encoding general-purpose region context.
  arXiv  Detail & Related papers  (2023-12-14T17:01:02Z)
• Location-Aware Self-Supervised Transformers [74.76585889813207]
  We propose to pretrain networks for semantic segmentation by predicting the relative location of image parts. We control the difficulty of the task by masking a subset of the reference patch features visible to those of the query. Our experiments show that this location-aware pretraining leads to representations that transfer competitively to several challenging semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2022-12-05T16:24:29Z)
• Dynamic Focus-aware Positional Queries for Semantic Segmentation [94.6834904076914]
  We propose a simple yet effective query design for semantic segmentation termed Dynamic Focus-aware Positional Queries. Our framework achieves SOTA performance and outperforms Mask2former by clear margins of 1.1%, 1.9%, and 1.1% single-scale mIoU with ResNet-50, Swin-T, and Swin-B backbones.
  arXiv  Detail & Related papers  (2022-04-04T05:16:41Z)
• Vision-Language Transformer and Query Generation for Referring Segmentation [39.01244764840372]
  We reformulate referring segmentation as a direct attention problem. We build a network with an encoder-decoder attention mechanism architecture that "queries" the given image with the language expression. Our approach is light-weight and achieves new state-of-the-art performance consistently on three referring segmentation datasets.
  arXiv  Detail & Related papers  (2021-08-12T07:24:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.