InsightSeg: Reusing Correction Insights for Guideline-Consistent Segmentation

• URL: http://arxiv.org/abs/2609.02002v1
• Date: Wed, 02 Sep 2026 02:22:35 GMT
• Title: InsightSeg: Reusing Correction Insights for Guideline-Consistent Segmentation
• Abstract summary: We introduce InsightSeg, an episodic memory mechanism that converts successful correction episodes into reusable, visually grounded insights.<n>Across and Cityscapes, InsightSeg improves both first-pass and final guideline-consistent segmentation performance while requiring fewer refinement steps.
• Score: 4.501547677532766
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Guideline-consistent semantic segmentation requires more than category recognition, as real-world labeling policies demand fine-grained, task-specific decisions. Recent multi-agent refinement systems improve compliance with such textual guidelines by detecting and correcting errors. However, they are stateless: feedback from the critiquing agent is discarded, causing the same guideline-specific mistakes to be repeatedly rediscovered and corrected across the dataset at the cost of additional refinement. We introduce InsightSeg, an episodic memory mechanism that converts successful correction episodes into reusable, visually grounded insights. A meta-analyzer distills each qualifying episode into directive natural-language insights and anchors them to the local image regions that caused the error using patch-level visual concept vectors. On subsequent images, these concepts are matched against dense patch embeddings to retrieve relevant insights, which condition the segmenting agent before making its first prediction. This shifts the system from correcting recurring errors to preventing them, improving segmentation quality before any refinement occurs. Across Waymo and Cityscapes, InsightSeg improves both first-pass and final guideline-consistent segmentation performance while requiring fewer refinement steps, demonstrating that multi-agent refinement can become more accurate and efficient by drawing on past correction experience.

Related papers

• Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generation [147.1183324267404]
  A local code edit propagates through layout, style, and component dependencies, correcting one visual mismatch while degrading regions that were previously faithful.<n>We present RubSE, a rubric-guided framework that uses rubrics to represent visual feedback as a structured visual-repair context.<n>RubSE substantially outperforms nave self-evolution in final-round and best-round settings, achieving more stable refinement and a higher trajectory-level performance ceiling.
  arXiv  Detail & Related papers  (2026-08-25T07:03:46Z)
• Unpaired Modality-Agnostic Generative Recommendation [26.342219762853187]
  UnpairGR learns a unified semantic-ID space from paired, image-only, and text-only observations.<n>Experiments on three benchmark datasets demonstrate that UnpairGR consistently improves recommendation performance.
  arXiv  Detail & Related papers  (2026-08-03T16:46:47Z)
• Beyond Supervised Clarification: Input Rewriting with LLMs for Dialogue Discourse Parsing [16.51635619165348]
  Rewriting inputs to improve frozen downstream models has become a common strategy in modern NLP pipelines.<n>In this work, we revisit this idea under realistic deployment conditions, where no clarification supervision is available.<n>A best-of-8 rewriting analysis reveals a practical ceiling: a large fraction of errors are not repairable through input alone.
  arXiv  Detail & Related papers  (2026-07-02T09:57:52Z)
• Speculative Rollback Correction for Quality-Diverse Web Agent Imitation [43.603850569331975]
  Training interactive web agents through imitation learning from expert trajectories has emerged as a highly effective approach.<n>We propose Speculative Rollback Correction (SRC), a branch-level imitation framework for resettable agent environments.
  arXiv  Detail & Related papers  (2026-06-10T08:56:27Z)
• Learn to Rank: Visual Attribution by Learning Importance Ranking [58.69028273772474]
  We propose a new approach to generate visual attribution maps for computer vision models.<n>Our method produces dense, pixel-level attributions in a single forward pass with optional, few-step gradient refinement.<n>Our experiments demonstrate consistent quantitative improvements and sharper, boundary-aligned explanations.
  arXiv  Detail & Related papers  (2026-04-07T12:53:22Z)
• SPEGC: Continual Test-Time Adaptation via Semantic-Prompt-Enhanced Graph Clustering for Medical Image Segmentation [12.0982298854338]
  Continual Test-Time Adaptation (CTTA) aims to enable pre-trained models to adapt to continuously changing unlabeled domains.<n>We propose a CTTA via Semantic-Prompt-Enhanced Graph Clustering (SPEGC) for medical image segmentation.
  arXiv  Detail & Related papers  (2026-03-12T03:22:43Z)
• What's Left Unsaid? Detecting and Correcting Misleading Omissions in Multimodal News Previews [31.373823254968315]
  Even when factually correct, social-media news previews can induce interpretation drift.<n>This covert harm is harder to detect than explicit misinformation yet remains underexplored.<n>We develop a multi-stage pipeline that disentangles and simulates preview-based versus context-based understanding.
  arXiv  Detail & Related papers  (2026-01-09T06:29:19Z)
• The Consistency Critic: Correcting Inconsistencies in Generated Images via Reference-Guided Attentive Alignment [105.31858867473845]
  ImageCritic can be integrated into an agent framework to automatically detect inconsistencies and correct them with multi-round and local editing.<n>In experiments, ImageCritic can effectively resolve detail-related issues in various customized generation scenarios, providing significant improvements over existing methods.
  arXiv  Detail & Related papers  (2025-11-25T18:40:25Z)
• Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction [58.51530390018909]
  Large Language Model based multi-agent systems excel at collaborative problem solving but remain brittle to cascading errors.<n>We present MASC, a metacognitive framework that endows MAS with real-time, unsupervised, step-level error detection and self-correction.
  arXiv  Detail & Related papers  (2025-10-16T05:35:37Z)
• Multimodal LLM-Guided Semantic Correction in Text-to-Image Diffusion [52.315729095824906]
  MLLM Semantic-Corrected Ping-Pong-Ahead Diffusion (PPAD) is a novel framework that introduces a Multimodal Large Language Model (MLLM) as a semantic observer during inference.<n>It performs real-time analysis on intermediate generations, identifies latent semantic inconsistencies, and translates feedback into controllable signals that actively guide the remaining denoising steps.<n>Extensive experiments demonstrate PPAD's significant improvements.
  arXiv  Detail & Related papers  (2025-05-26T14:42:35Z)
• SINDER: Repairing the Singular Defects of DINOv2 [61.98878352956125]
  Vision Transformer models trained on large-scale datasets often exhibit artifacts in the patch token they extract. We propose a novel fine-tuning smooth regularization that rectifies structural deficiencies using only a small dataset.
  arXiv  Detail & Related papers  (2024-07-23T20:34:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.