Generative Semantic Segmentation via an Observable Semantic-Image Interface and Hierarchical Generator Evidence Alignment

• URL: http://arxiv.org/abs/2608.11537v1
• Date: Wed, 12 Aug 2026 01:00:04 GMT
• Title: Generative Semantic Segmentation via an Observable Semantic-Image Interface and Hierarchical Generator Evidence Alignment
• Abstract summary: Generative semantic segmentation exposes structured predictions as images, but direct color decoding is susceptible to color drift and boundary mixing.<n>We present Semantic Prism, a conditional semantic-image generation-and-refinement framework with inference.
• Score: 1.6199400106794555
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative semantic segmentation exposes structured predictions as images, but direct color decoding is susceptible to color drift and boundary mixing, whereas latent-feature decoders that predict a separate output distribution may relegate the rendered image to an intermediate visualization. We present Semantic Prism, a conditional semantic-image generation-and-refinement framework with deterministic inference. A diffusion-distilled one-step generator renders a semantic RGB image; per-pixel distances from the rendered colors to a fixed class-color codebook define an explicit probabilistic interface. Hierarchical Generator Evidence Alignment spatially aligns multi-level generator features and uses a zero-initialized output projection to predict an additive residual in the interface logit space, retaining the image-defined interface as the reference for the final distribution. The interface and refined distributions further enable Contextual Interface--Hierarchy Disagreement (C-IHD), a fixed readout for ranking remaining pixel errors without an auxiliary predictor or additional forward pass. On the 500-image Cityscapes validation set, Semantic Prism achieves 72.07% mean intersection over union, 11.39 mIoU points above direct-interface decoding, with 0.41% expected calibration error. Matched-capacity ablations over three seeds support the benefit of jointly aligned multi-level evidence. A separately trained model attains 62.22% mIoU on BDD100K, while the Cityscapes-trained model reaches 46.89\% mIoU under source-frozen transfer to the Adverse Conditions Dataset with Correspondences, without target-domain adaptation. Across all three datasets, C-IHD consistently improves the area under the precision--recall curve for pixel-error ranking over maximum softmax probability on the same segmentation predictions; on ACDC, it raises AUPR from 0.6580 to 0.7557.

Related papers

• SeGDeP: Semantic- and Geometric-Aware Decoupled Prompts for Reasoning Segmentation [51.46828526392219]
  Reasoning segmentation converts an implicit linguistic conclusion into a precise mask.<n>Existing MLLM-segmenter interfaces either use a special trigger or compress both signals into one context.<n>We present SeGDeP, an explicit what-where interface.
  arXiv  Detail & Related papers  (2026-09-08T15:11:32Z)
• Hierarchical Adaptive Feature Refinement Network for VHR Remote Sensing Image Segmentation [15.24999553337715]
  We present HAFR-Net, a progressive refinement framework that adaptively organizes and refines hierarchical representations.<n>Under a matched Swin-B training and single-scale inference protocol, HAFR-Net attains 84.12%, 87.86%, 55.17%, and 67.70% mIoU.
  arXiv  Detail & Related papers  (2026-08-16T09:26:58Z)
• SC-Diff: Semantically Calibrated Diffusion for Visible-to-Infrared Image Translation [52.62632888060867]
  Visible-to-infrared image translation provides a practical way to expand infrared training data using abundant visible images.<n>Existing diffusion-based methods typically use semantic priors only as external conditions.<n>We propose SC-Diff, a semantically latent diffusion framework that uses semantic priors for both conditional guidance and internal self-attention calibration.
  arXiv  Detail & Related papers  (2026-08-09T07:57:40Z)
• Where Does Generative Difficulty Reside? An Empirical Study of Target Representations [62.54876287800644]
  The target representation defines the distribution an image generator must learn, yet it is often treated as an interchangeable interface.<n>We study raw pixels, SD-VAE latents and DINOv2 as well as MAE representation-autoencoder features within a unified masked autoregressive rectified-flow model.<n>Our results show that compression, reconstruction fidelity, token dimensionality, and visible semantic clustering do not individually predict generative behavior.
  arXiv  Detail & Related papers  (2026-08-01T12:32:31Z)
• CrossFlow: One-Step Generation Across Latent and Pixel Spaces [61.08358796319721]
  CrossFlow is a cross-space flow formulation that maps noisy latent inputs directly to pixel-space images.<n>On class-conditional ImageNet-1k at $256times256$, CrossFlow-XL achieves 1.62 FID with one function evaluation.
  arXiv  Detail & Related papers  (2026-06-18T09:10:28Z)
• Surflo: Consistent 3D Surface Flow Model with Global State [57.57234680235885]
  Surflo decodes oriented 3D surface points by independently transporting them from noise onto the surface via flow matching.<n>To suppress the local inconsistencies inherent to independent per-point decoding, an inference-time guidance term correlates nearby points by injecting a photometric gradient.<n>Surflo matches or surpasses feed-forward baselines on surface metrics, runs an order of magnitude faster than optimization-based methods that require hundreds of views, and is the only feed-forward approach to combine a global latent with arbitrary-resolution decoding.
  arXiv  Detail & Related papers  (2026-06-11T17:48:38Z)
• iSAGE: A Human-in-the-Loop Framework for Remote Sensing Semantic Segmentation via Sparse Point Supervision [0.1749935196721634]
  Existing human-in-the-loop frameworks expand sparse clicks into dense supervision via auxiliary machinery.<n>This paper hypothesizes that expert clicks targeting confident model errors, not arbitrary pixels, suffice to match dense supervision.<n>iSAGE is the only iterative human-in-the-loop framework operating without auxiliary machinery.
  arXiv  Detail & Related papers  (2026-06-08T20:09:35Z)
• CMAP: Cross-Modal Adaptive Prompting for Multi-Domain Task-Incremental Learning [3.1368611610608856]
  Text-space task routing replaces visual Gaussian matching with cosine similarity to frozen CLIP text prototypes.<n>Visual-textual confidence replaces single-Gaussian class modeling with K-means visual prototypes and cross-modal alignment scores under task-calibrated thresholds.<n>On the MTIL benchmark spanning 118.7% and 1201 classes, our method achieves 74.2% Transfer, 80.5% Average, and 8 Last under Order-I.
  arXiv  Detail & Related papers  (2026-05-25T11:09:48Z)
• ASCNet: Asymmetric Sampling Correction Network for Infrared Image Destriping [25.816984064145906]
  We propose a novel infrared image destriping method called Asymmetric Sampling Correction Network (ASCNet)<n>Our ASCNet consists of three core elements: Residual Haar Discrete Wavelet Transform (RHDWT), Pixel Shuffle (PS), and Column Non-uniformity Correction Module (CNCM)
  arXiv  Detail & Related papers  (2024-01-28T06:23:55Z)
• Zero-shot spatial layout conditioning for text-to-image diffusion models [52.24744018240424]
  Large-scale text-to-image diffusion models have significantly improved the state of the art in generative image modelling. We consider image generation from text associated with segments on the image canvas, which combines an intuitive natural language interface with precise spatial control over the generated content. We propose ZestGuide, a zero-shot segmentation guidance approach that can be plugged into pre-trained text-to-image diffusion models.
  arXiv  Detail & Related papers  (2023-06-23T19:24:48Z)
• Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation [59.94819184452694]
  Depth information has proven to be a useful cue in the semantic segmentation of RGBD images for providing a geometric counterpart to the RGB representation. Most existing works simply assume that depth measurements are accurate and well-aligned with the RGB pixels and models the problem as a cross-modal feature fusion. In this paper, we propose a unified and efficient Crossmodality Guided to not only effectively recalibrate RGB feature responses, but also to distill accurate depth information via multiple stages and aggregate the two recalibrated representations alternatively.
  arXiv  Detail & Related papers  (2020-07-17T18:35:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.