Open-World Semantic Segmentation with Sensitivity Modeling

• URL: http://arxiv.org/abs/2608.08308v1
• Date: Sat, 08 Aug 2026 19:37:02 GMT
• Title: Open-World Semantic Segmentation with Sensitivity Modeling
• Abstract summary: Open-world vision systems must recognize known categories and detect unseen or anomalous content.<n>We extend a dual-decoder baseline with a third, complementary decoder within a unified encoder-decoder design.<n>Our method improves anomaly segmentation and novel-class discovery while maintaining competitive closed-set accuracy.
• Score: 6.086784305572313
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern vision systems must operate in "open-world" settings, where models must recognize known categories and detect unseen or anomalous content. Conventional semantic segmentation models operate under a "closed-world" assumption, often producing overconfident misclassifications on novel content. We address open-world semantic segmentation, the joint task of segmenting known classes while detecting and grouping novel or anomalous content without additional supervision, by extending a dual-decoder baseline with a third, complementary decoder within a unified encoder-decoder design. The first decoder performs closed-set segmentation using Gaussian prototypes for known categories. The second uses contrastive feature learning to isolate unknown regions in embedding space. The third, our key contribution, is a sensitivity decoder that captures fine-grained texture irregularities and activation instabilities indicative of semantic uncertainty, which neither semantic prototypes nor contrastive norms can reliably detect. The three decoders provide genuinely complementary signals: class-level OOD distance in logit space, global feature energy in embedding space, and local activation instability across encoder scales. Experiments on Cityscapes and BDD-Anomaly show that our method improves anomaly segmentation and novel-class discovery while maintaining competitive closed-set accuracy, with gains of +2.4% AUROC and a 2.5 pp. reduction in FPR@95TPR on BDD-Anomaly over the baseline.

Related papers

• REAL-OW: Rehearsal-free Open World Object Detection with Low-Rank Adaptation and Dual-Stage Objectness Modeling [56.888597798431704]
  Open-World Object Detection (OWOD) requires detectors to identify previously unseen objects as unknown and incrementally incorporate them into the set of known categories.<n>Existing frameworks rely heavily on exemplar replay to mitigate catastrophic forgetting, but in some real applications, storing raw data conflicts with data access restrictions and leads to data exposure risks.<n>We propose REAL-OW, a novel rehearsal-free framework that decouples incremental knowledge through a collaborative adapter architecture.
  arXiv  Detail & Related papers  (2026-07-03T06:38:54Z)
• ShortcutBreaker: Low-Rank Noisy Bottleneck with Global Perturbation Attention for Multi-Class Unsupervised Anomaly Detection [59.89803740308262]
  ShortcutBreaker is a novel unified feature-reconstruction framework for MUAD tasks.<n>It features two key innovations to address the issue of shortcuts.<n>The proposed method achieves a remarkable image-level AUROC of 99.8%, 98.9%, 90.6%, and 87.8% on four datasets.
  arXiv  Detail & Related papers  (2025-10-21T06:51:30Z)
• AD-DINOv3: Enhancing DINOv3 for Zero-Shot Anomaly Detection with Anomaly-Aware Calibration [12.642531824086639]
  Zero-Shot Anomaly Detection (ZSAD) seeks to identify anomalies from arbitrary novel categories.<n>Recent vision foundation models such as DINOv3 have demonstrated strong transferable representation capabilities.<n>We introduce AD-DINOv3, a novel vision-language multimodal framework designed for ZSAD.
  arXiv  Detail & Related papers  (2025-09-17T15:29:25Z)
• NeurNCD: Novel Class Discovery via Implicit Neural Representation [4.498082064000176]
  NeurNCD is a versatile and data-efficient framework for novel class discovery.<n>Our framework achieves superior segmentation performance in both open and closed-world settings.<n>Our method significantly outperforms state-of-the-art approaches on the NYUv2 and Replica datasets.
  arXiv  Detail & Related papers  (2025-06-06T16:43:34Z)
• Crane: Context-Guided Prompt Learning and Attention Refinement for Zero-Shot Anomaly Detection [50.343419243749054]
  Anomaly detection is critical in fields such as medical diagnostics and industrial defect detection.<n> CLIP's coarse-grained image-text alignment limits localization and detection performance for fine-grained anomalies.<n>Crane improves the state-of-the-art ZSAD from 2% to 28%, at both image and pixel levels, while remaining competitive in inference speed.
  arXiv  Detail & Related papers  (2025-04-15T10:42:25Z)
• SNE-RoadSegV2: Advancing Heterogeneous Feature Fusion and Fallibility Awareness for Freespace Detection [29.348921424716057]
  This paper presents a novel heterogeneous feature fusion block, comprising a holistic attention module, a heterogeneous feature contrast descriptor, and an affinity-weighted feature recalibrator. It incorporates both inter-scale and intra-scale skip connections into the decoder architecture while eliminating redundant ones, leading to both improved accuracy and computational efficiency. It introduces two fallibility-aware loss functions that separately focus on semantic-transition and depth-inconsistent regions, collectively contributing to greater supervision during model training.
  arXiv  Detail & Related papers  (2024-02-29T07:20:02Z)
• Triple-View Knowledge Distillation for Semi-Supervised Semantic Segmentation [54.23510028456082]
  We propose a Triple-view Knowledge Distillation framework, termed TriKD, for semi-supervised semantic segmentation. The framework includes the triple-view encoder and the dual-frequency decoder.
  arXiv  Detail & Related papers  (2023-09-22T01:02:21Z)
• GLENet: Boosting 3D Object Detectors with Generative Label Uncertainty Estimation [70.75100533512021]
  In this paper, we formulate the label uncertainty problem as the diversity of potentially plausible bounding boxes of objects. We propose GLENet, a generative framework adapted from conditional variational autoencoders, to model the one-to-many relationship between a typical 3D object and its potential ground-truth bounding boxes with latent variables. The label uncertainty generated by GLENet is a plug-and-play module and can be conveniently integrated into existing deep 3D detectors.
  arXiv  Detail & Related papers  (2022-07-06T06:26:17Z)
• The KFIoU Loss for Rotated Object Detection [115.334070064346]
  In this paper, we argue that one effective alternative is to devise an approximate loss who can achieve trend-level alignment with SkewIoU loss. Specifically, we model the objects as Gaussian distribution and adopt Kalman filter to inherently mimic the mechanism of SkewIoU. The resulting new loss called KFIoU is easier to implement and works better compared with exact SkewIoU.
  arXiv  Detail & Related papers  (2022-01-29T10:54:57Z)
• Omni-supervised Point Cloud Segmentation via Gradual Receptive Field Component Reasoning [41.83979510282989]
  We bring the first omni-scale supervision method to point cloud segmentation via the proposed gradual Receptive Field Component Reasoning (RFCR) Our method brings new state-of-the-art performances for S3DIS as well as Semantic3D and ranks the 1st in the ScanNet benchmark among all the point-based methods.
  arXiv  Detail & Related papers  (2021-05-21T08:32:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.