Road-Aware Anomaly Segmentation with Query-Guided Polygons and CLIP in Autonomous Driving

• URL: http://arxiv.org/abs/2607.04304v1
• Date: Sun, 05 Jul 2026 13:43:31 GMT
• Title: Road-Aware Anomaly Segmentation with Query-Guided Polygons and CLIP in Autonomous Driving
• Abstract summary: We present a lightweight, postprocessing, road-aware anomaly segmentation framework that requires no retraining, no OOD data, and no auxiliary supervision.<n>Our approach builds on a mask transformer-based segmentation network by exploiting query-level mask confidence.<n> Evaluation on three public benchmarks-Fishyscapes, SMIYC, and RoadAnomaly-shows consistently strong performance.
• Score: 0.5371337604556311
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional semantic segmentation models operate under a closed-set assumption and struggle to recognize unknown or unexpected objects-an essential capability for autonomous driving. As a result, such models often misclassify or overlook out-of-distribution (OOD) road anomalies, posing safety risks in open-world environments. We present a lightweight, postprocessing, road-aware anomaly segmentation framework that requires no retraining, no OOD data, and no auxiliary supervision. Our approach builds on a mask transformer-based segmentation network by exploiting query-level mask confidence and deriving a polygonal road prior to detect gap regions that may correspond to anomalies. To further suppress false positives, we introduce a CLIP-based zero-shot semantic filtering module using in-distribution prompts, with optional generalized OOD prompts. By jointly leveraging spatial priors and semantic verification, our framework produces robust and interpretable anomaly predictions. Evaluation on three public benchmarks-Fishyscapes, SMIYC, and RoadAnomaly-shows consistently strong performance. In particular, our method outperforms the training-free baseline Maskomaly on most metrics and achieves the highest AP on Fishyscapes LostAndFound. These results demonstrate the practicality and deployability of our approach for real-world autonomous driving systems.

Related papers

• Vision-Language Feature Alignment for Road Anomaly Segmentation [38.2615882515309]
  We propose a vision-language anomaly segmentation framework that incorporates semantic priors from pre-trained Vision-Language Models (VLMs)<n>Specifically, we design a prompt learning-driven alignment module that adapts Mask2Forme's visual features to CLIP text embeddings of known categories.<n>At inference time, we introduce a multi-source inference strategy that integrates text-guided similarity, CLIP-based image-text similarity and detector confidence.
  arXiv  Detail & Related papers  (2026-03-01T10:17:00Z)
• Unsupervised Learning for Detection of Rare Driving Scenarios [1.5399429731150376]
  This research explores an unsupervised learning framework for detecting rare and extreme driving scenarios.<n>We leverage the recently proposed Deep Isolation Forest (DIF) to identify non-linear and complex anomalies.<n>Our results demonstrate that the proposed approach effectively identifies rare and hazardous driving scenarios.
  arXiv  Detail & Related papers  (2025-12-29T16:35:20Z)
• Out-of-Distribution Segmentation via Wasserstein-Based Evidential Uncertainty [22.45768202733885]
  We present an evidence segmentation framework using a Wasserstein loss, which captures distributional distances while respecting the probability simplex geometry.<n>Our approach leads to improved OOD segmentation performance compared to uncertainty-based approaches.
  arXiv  Detail & Related papers  (2025-12-12T08:36:59Z)
• Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving [55.13109926181247]
  We introduce ReflectDrive, a learning-based framework that integrates a reflection mechanism for safe trajectory generation via discrete diffusion.<n>Central to our approach is a safety-aware reflection mechanism that performs iterative self-correction without gradient.<n>Our method begins with goal-conditioned trajectory generation to model multi-modal driving behaviors.
  arXiv  Detail & Related papers  (2025-09-24T13:35:15Z)
• Segmenting Objectiveness and Task-awareness Unknown Region for Autonomous Driving [46.70405993442064]
  We propose a novel framework termed Segmenting Objectiveness and Task-Awareness (SOTA) for autonomous driving scenes.<n>SOTA enhances the segmentation of objectiveness through a Semantic Fusion Block (SFB) and filters anomalies irrelevant to road navigation tasks.
  arXiv  Detail & Related papers  (2025-04-27T10:08:54Z)
• Prior2Former -- Evidential Modeling of Mask Transformers for Assumption-Free Open-World Panoptic Segmentation [74.55677741919035]
  We propose Prior2Former (P2F), the first approach for segmentation vision transformers rooted in evidential learning.<n>P2F extends the mask vision transformer architecture by incorporating a Beta prior for computing model uncertainty in pixel-wise binary mask assignments.<n>Unlike most segmentation models addressing unknown classes, P2F operates without access to OOD data samples or contrastive training on void (i.e., unlabeled) classes.
  arXiv  Detail & Related papers  (2025-04-07T08:53:14Z)
• Multi-modality Anomaly Segmentation on the Road [39.96773593485109]
  Current uni-modal anomaly segmentation frameworks tend to produce high anomaly scores for non-anomalous regions in images.<n>We develop a multi-modal uncertainty-based anomaly segmentation framework, named MMRAS+, for autonomous driving systems.<n>Indeed, MMRAS+ is the first multi-modal anomaly segmentation solution for autonomous driving.
  arXiv  Detail & Related papers  (2025-03-22T09:55:42Z)
• Unsupervised Self-Driving Attention Prediction via Uncertainty Mining and Knowledge Embedding [51.8579160500354]
  We propose an unsupervised way to predict self-driving attention by uncertainty modeling and driving knowledge integration. Results show equivalent or even more impressive performance compared to fully-supervised state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-03-17T00:28:33Z)
• Traj-MAE: Masked Autoencoders for Trajectory Prediction [69.7885837428344]
  Trajectory prediction has been a crucial task in building a reliable autonomous driving system by anticipating possible dangers. We propose an efficient masked autoencoder for trajectory prediction (Traj-MAE) that better represents the complicated behaviors of agents in the driving environment. Our experimental results in both multi-agent and single-agent settings demonstrate that Traj-MAE achieves competitive results with state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-03-12T16:23:27Z)
• Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts? [104.04999499189402]
  Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment. We propose an uncertainty-aware planning method, called emphrobust imitative planning (RIP) Our method can detect and recover from some distribution shifts, reducing the overconfident and catastrophic extrapolations in OOD scenes. We introduce an autonomous car novel-scene benchmark, textttCARNOVEL, to evaluate the robustness of driving agents to a suite of tasks with distribution shifts.
  arXiv  Detail & Related papers  (2020-06-26T11:07:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.