Open-world Semantic Segmentation for LIDAR Point Clouds

• URL: http://arxiv.org/abs/2207.01452v1
• Date: Mon, 4 Jul 2022 14:40:35 GMT
• Title: Open-world Semantic Segmentation for LIDAR Point Clouds
• Authors: Jun Cen, Peng Yun, Shiwei Zhang, Junhao Cai, Di Luan, Michael Yu Wang, Ming Liu, Mingqian Tang
• Abstract summary: We propose an open-world semantic segmentation task for LIDAR point clouds. It aims to identify both old and novel classes using open-set semantic segmentation. It also gradually incorporate novel objects into the existing knowledge base using incremental learning.
• Score: 18.45831801175225
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Current methods for LIDAR semantic segmentation are not robust enough for real-world applications, e.g., autonomous driving, since it is closed-set and static. The closed-set assumption makes the network only able to output labels of trained classes, even for objects never seen before, while a static network cannot update its knowledge base according to what it has seen. Therefore, in this work, we propose the open-world semantic segmentation task for LIDAR point clouds, which aims to 1) identify both old and novel classes using open-set semantic segmentation, and 2) gradually incorporate novel objects into the existing knowledge base using incremental learning without forgetting old classes. For this purpose, we propose a REdundAncy cLassifier (REAL) framework to provide a general architecture for both the open-set semantic segmentation and incremental learning problems. The experimental results show that REAL can simultaneously achieves state-of-the-art performance in the open-set semantic segmentation task on the SemanticKITTI and nuScenes datasets, and alleviate the catastrophic forgetting problem with a large margin during incremental learning.

Related papers

• Exploiting the Semantic Knowledge of Pre-trained Text-Encoders for Continual Learning [70.64617500380287]
  Continual learning allows models to learn from new data while retaining previously learned knowledge. The semantic knowledge available in the label information of the images, offers important semantic information that can be related with previously acquired knowledge of semantic classes. We propose integrating semantic guidance within and across tasks by capturing semantic similarity using text embeddings.
  arXiv  Detail & Related papers  (2024-08-02T07:51:44Z)
• Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised Semantic Segmentation [79.05949524349005]
  We propose AuxSegNet+, a weakly supervised auxiliary learning framework to explore the rich information from saliency maps. We also propose a cross-task affinity learning mechanism to learn pixel-level affinities from the saliency and segmentation feature maps.
  arXiv  Detail & Related papers  (2024-03-02T10:03:21Z)
• Open-Vocabulary Camouflaged Object Segmentation [66.94945066779988]
  We introduce a new task, open-vocabulary camouflaged object segmentation (OVCOS) We construct a large-scale complex scene dataset (textbfOVCamo) containing 11,483 hand-selected images with fine annotations and corresponding object classes. By integrating the guidance of class semantic knowledge and the supplement of visual structure cues from the edge and depth information, the proposed method can efficiently capture camouflaged objects.
  arXiv  Detail & Related papers  (2023-11-19T06:00:39Z)
• SegPrompt: Boosting Open-world Segmentation via Category-level Prompt Learning [49.17344010035996]
  Open-world instance segmentation (OWIS) models detect unknown objects in a class-agnostic manner. Previous OWIS approaches completely erase category information during training to keep the model's ability to generalize to unknown objects. We propose a novel training mechanism termed SegPrompt that uses category information to improve the model's class-agnostic segmentation ability.
  arXiv  Detail & Related papers  (2023-08-12T11:25:39Z)
• Coreset Sampling from Open-Set for Fine-Grained Self-Supervised Learning [10.57079240576682]
  We introduce a novel Open-Set Self-Supervised Learning problem under the assumption that a large-scale unlabeled open-set is available. In our problem setup, it is crucial to consider the distribution mismatch between the open-set and target dataset. We demonstrate that SimCore significantly improves representation learning performance through extensive experimental settings.
  arXiv  Detail & Related papers  (2023-03-20T13:38:29Z)
• ElC-OIS: Ellipsoidal Clustering for Open-World Instance Segmentation on LiDAR Data [13.978966783993146]
  Open-world Instance (OIS) is a challenging task that aims to accurately segment every object instance appearing in the current observation. This is important for safety-critical applications such as robust autonomous navigation. We present a flexible and effective OIS framework for LiDAR point cloud that can accurately segment both known and unknown instances.
  arXiv  Detail & Related papers  (2023-03-08T03:22:11Z)
• Learning from Mistakes: Self-Regularizing Hierarchical Representations in Point Cloud Semantic Segmentation [15.353256018248103]
  LiDAR semantic segmentation has gained attention to accomplish fine-grained scene understanding. We present a coarse-to-fine setup that LEArns from classification mistaKes (LEAK) derived from a standard model. Our LEAK approach is very general and can be seamlessly applied on top of any segmentation architecture.
  arXiv  Detail & Related papers  (2023-01-26T14:52:30Z)
• Continual Coarse-to-Fine Domain Adaptation in Semantic Segmentation [22.366638308792734]
  Deep neural networks are typically trained in a single shot for a specific task and data distribution. In real world settings both the task and the domain of application can change. We introduce the novel task of coarse-to-fine learning of semantic segmentation architectures in presence of domain shift.
  arXiv  Detail & Related papers  (2022-01-18T13:31:19Z)
• Deep Metric Learning for Open World Semantic Segmentation [12.617115020561789]
  Close-set semantic segmentation networks have limited ability to detect out-of-distribution (OOD) objects. We propose an open world semantic segmentation system that includes two modules. We adopt the Deep Metric Learning Network (DMLNet) with contrastive clustering to implement open-set semantic segmentation.
  arXiv  Detail & Related papers  (2021-08-10T10:15:57Z)
• Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentation [88.49669148290306]
  We propose a novel weakly supervised multi-task framework called AuxSegNet to leverage saliency detection and multi-label image classification as auxiliary tasks. Inspired by their similar structured semantics, we also propose to learn a cross-task global pixel-level affinity map from the saliency and segmentation representations. The learned cross-task affinity can be used to refine saliency predictions and propagate CAM maps to provide improved pseudo labels for both tasks.
  arXiv  Detail & Related papers  (2021-07-25T11:39:58Z)
• Exemplar-Based Open-Set Panoptic Segmentation Network [79.99748041746592]
  We extend panoptic segmentation to the open-world and introduce an open-set panoptic segmentation (OPS) task. We investigate the practical challenges of the task and construct a benchmark on top of an existing dataset, COCO. We propose a novel exemplar-based open-set panoptic segmentation network (EOPSN) inspired by exemplar theory.
  arXiv  Detail & Related papers  (2021-05-18T07:59:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.