A Novel Decomposed Feature-Oriented Framework for Open-Set Semantic Segmentation on LiDAR Data

• URL: http://arxiv.org/abs/2503.11097v1
• Date: Fri, 14 Mar 2025 05:40:05 GMT
• Title: A Novel Decomposed Feature-Oriented Framework for Open-Set Semantic Segmentation on LiDAR Data
• Authors: Wenbang Deng, Xieyuanli Chen, Qinghua Yu, Yunze He, Junhao Xiao, Huimin Lu,
• Abstract summary: We propose a feature-oriented framework for open-set semantic segmentation on LiDAR data.<n>We design a dual-decoder network to simultaneously perform closed-set semantic segmentation and generate distinctive features for unknown objects.<n>By integrating the results of close-set semantic segmentation and anomaly detection, we achieve effective feature-driven LiDAR open-set semantic segmentation.
• Score: 6.427051055902494
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Semantic segmentation is a key technique that enables mobile robots to understand and navigate surrounding environments autonomously. However, most existing works focus on segmenting known objects, overlooking the identification of unknown classes, which is common in real-world applications. In this paper, we propose a feature-oriented framework for open-set semantic segmentation on LiDAR data, capable of identifying unknown objects while retaining the ability to classify known ones. We design a decomposed dual-decoder network to simultaneously perform closed-set semantic segmentation and generate distinctive features for unknown objects. The network is trained with multi-objective loss functions to capture the characteristics of known and unknown objects. Using the extracted features, we introduce an anomaly detection mechanism to identify unknown objects. By integrating the results of close-set semantic segmentation and anomaly detection, we achieve effective feature-driven LiDAR open-set semantic segmentation. Evaluations on both SemanticKITTI and nuScenes datasets demonstrate that our proposed framework significantly outperforms state-of-the-art methods. The source code will be made publicly available at https://github.com/nubot-nudt/DOSS.

Related papers

• OW-Rep: Open World Object Detection with Instance Representation Learning [1.8749305679160366]
  Open World Object Detection (OWOD) addresses realistic scenarios where unseen object classes emerge. We extend the OWOD framework to jointly detect unknown objects and learn semantically rich instance embeddings.
  arXiv  Detail & Related papers  (2024-09-24T13:13:34Z)
• Learning Spatial-Semantic Features for Robust Video Object Segmentation [108.045326229865]
  We propose a robust video object segmentation framework equipped with spatial-semantic features and discriminative object queries. We show that the proposed method set a new state-of-the-art performance on multiple datasets.
  arXiv  Detail & Related papers  (2024-07-10T15:36:00Z)
• Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos [63.94040814459116]
  Self-supervised methods have shown remarkable progress in learning high-level semantics and low-level temporal correspondence. We propose a novel semantic-aware masked slot attention on top of the fused semantic features and correspondence maps. We adopt semantic- and instance-level temporal consistency as self-supervision to encourage temporally coherent object-centric representations.
  arXiv  Detail & Related papers  (2023-08-19T09:12:13Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• SalienDet: A Saliency-based Feature Enhancement Algorithm for Object Detection for Autonomous Driving [160.57870373052577]
  We propose a saliency-based OD algorithm (SalienDet) to detect unknown objects. Our SalienDet utilizes a saliency-based algorithm to enhance image features for object proposal generation. We design a dataset relabeling approach to differentiate the unknown objects from all objects in training sample set to achieve Open-World Detection.
  arXiv  Detail & Related papers  (2023-05-11T16:19:44Z)
• ElC-OIS: Ellipsoidal Clustering for Open-World Instance Segmentation on LiDAR Data [13.978966783993146]
  Open-world Instance (OIS) is a challenging task that aims to accurately segment every object instance appearing in the current observation. This is important for safety-critical applications such as robust autonomous navigation. We present a flexible and effective OIS framework for LiDAR point cloud that can accurately segment both known and unknown instances.
  arXiv  Detail & Related papers  (2023-03-08T03:22:11Z)
• RbA: Segmenting Unknown Regions Rejected by All [1.3381749415517021]
  We show that the object queries in mask classification tend to behave like one vs all classifiers. We propose a novel outlier scoring function called RbA by defining the event of being an outlier as being rejected by all known classes. Our experiments show that mask classification improves the performance of the existing outlier detection methods, and the best results are achieved with the proposed RbA.
  arXiv  Detail & Related papers  (2022-11-25T18:50:04Z)
• Open-Set Object Detection Using Classification-free Object Proposal and Instance-level Contrastive Learning [25.935629339091697]
  Open-set object detection (OSOD) is a promising direction to handle the problem consisting of two subtasks: objects and background separation, and open-set object classification. We present Openset RCNN to address the challenging OSOD. We show that our Openset RCNN can endow the robot with an open-set perception ability to support robotic rearrangement tasks in cluttered environments.
  arXiv  Detail & Related papers  (2022-11-21T15:00:04Z)
• Open-world Semantic Segmentation for LIDAR Point Clouds [18.45831801175225]
  We propose an open-world semantic segmentation task for LIDAR point clouds. It aims to identify both old and novel classes using open-set semantic segmentation. It also gradually incorporate novel objects into the existing knowledge base using incremental learning.
  arXiv  Detail & Related papers  (2022-07-04T14:40:35Z)
• SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation [111.61261419566908]
  Deep neural networks (DNNs) are usually trained on a closed set of semantic classes. They are ill-equipped to handle previously-unseen objects. detecting and localizing such objects is crucial for safety-critical applications such as perception for automated driving.
  arXiv  Detail & Related papers  (2021-04-30T07:58:19Z)
• Target-Aware Object Discovery and Association for Unsupervised Video Multi-Object Segmentation [79.6596425920849]
  This paper addresses the task of unsupervised video multi-object segmentation. We introduce a novel approach for more accurate and efficient unseen-temporal segmentation. We evaluate the proposed approach on DAVIS$_17$ and YouTube-VIS, and the results demonstrate that it outperforms state-of-the-art methods both in segmentation accuracy and inference speed.
  arXiv  Detail & Related papers  (2021-04-10T14:39:44Z)
• Synthesizing the Unseen for Zero-shot Object Detection [72.38031440014463]
  We propose to synthesize visual features for unseen classes, so that the model learns both seen and unseen objects in the visual domain. We use a novel generative model that uses class-semantics to not only generate the features but also to discriminatively separate them.
  arXiv  Detail & Related papers  (2020-10-19T12:36:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.