Improving Open-Set Semantic Segmentation in 3D Point Clouds by Conditional Channel Capacity Maximization: Preliminary Results

• URL: http://arxiv.org/abs/2505.11521v1
• Date: Fri, 09 May 2025 04:12:26 GMT
• Title: Improving Open-Set Semantic Segmentation in 3D Point Clouds by Conditional Channel Capacity Maximization: Preliminary Results
• Authors: Wang Fang, Shirin Rahimi, Olivia Bennett, Sophie Carter, Mitra Hassani, Xu Lan, Omid Javadi, Lucas Mitchell,
• Abstract summary: We propose a plug and play framework for Open-Set Semantic (O3S)<n>By modeling the segmentation pipeline as a conditional Markov chain, we derive a novel regularizer term dubbed Conditional Channel Capacity Maximization (3CM)<n>We show that 3CM encourages the encoder to retain richer, label-dependent features, thereby enhancing the network's ability to distinguish and segment previously unseen categories.
• Score: 1.1328543389752008
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Point-cloud semantic segmentation underpins a wide range of critical applications. Although recent deep architectures and large-scale datasets have driven impressive closed-set performance, these models struggle to recognize or properly segment objects outside their training classes. This gap has sparked interest in Open-Set Semantic Segmentation (O3S), where models must both correctly label known categories and detect novel, unseen classes. In this paper, we propose a plug and play framework for O3S. By modeling the segmentation pipeline as a conditional Markov chain, we derive a novel regularizer term dubbed Conditional Channel Capacity Maximization (3CM), that maximizes the mutual information between features and predictions conditioned on each class. When incorporated into standard loss functions, 3CM encourages the encoder to retain richer, label-dependent features, thereby enhancing the network's ability to distinguish and segment previously unseen categories. Experimental results demonstrate effectiveness of proposed method on detecting unseen objects. We further outline future directions for dynamic open-world adaptation and efficient information-theoretic estimation.

Related papers

• BFANet: Revisiting 3D Semantic Segmentation with Boundary Feature Analysis [33.53327976669034]
  We revisit 3D semantic segmentation through a more granular lens, shedding light on subtle complexities that are typically overshadowed by broader performance metrics.<n>We introduce an innovative 3D semantic segmentation network called BFANet that incorporates detailed analysis of semantic boundary features.
  arXiv  Detail & Related papers  (2025-03-16T15:13:11Z)
• Class-Imbalanced Semi-Supervised Learning for Large-Scale Point Cloud Semantic Segmentation via Decoupling Optimization [64.36097398869774]
  Semi-supervised learning (SSL) has been an active research topic for large-scale 3D scene understanding. The existing SSL-based methods suffer from severe training bias due to class imbalance and long-tail distributions of the point cloud data. We introduce a new decoupling optimization framework, which disentangles feature representation learning and classifier in an alternative optimization manner to shift the bias decision boundary effectively.
  arXiv  Detail & Related papers  (2024-01-13T04:16:40Z)
• 3D Open-Vocabulary Panoptic Segmentation with 2D-3D Vision-Language Distillation [40.49322398635262]
  We propose the first method to tackle 3D open-vocabulary panoptic segmentation. Our model takes advantage of the fusion between learnable LiDAR features and dense frozen vision CLIP features. We propose two novel loss functions: object-level distillation loss and voxel-level distillation loss.
  arXiv  Detail & Related papers  (2024-01-04T18:39:32Z)
• Multi-modality Affinity Inference for Weakly Supervised 3D Semantic Segmentation [47.81638388980828]
  We propose a simple yet effective scene-level weakly supervised point cloud segmentation method with a newly introduced multi-modality point affinity inference module. Our method outperforms the state-of-the-art by 4% to 6% mIoU on the ScanNet and S3DIS benchmarks.
  arXiv  Detail & Related papers  (2023-12-27T14:01:35Z)
• A Data-efficient Framework for Robotics Large-scale LiDAR Scene Parsing [10.497309421830671]
  Existing state-of-the-art 3D point clouds understanding methods only perform well in a fully supervised manner. This work presents a general and simple framework to tackle point clouds understanding when labels are limited.
  arXiv  Detail & Related papers  (2023-12-03T02:38:51Z)
• Generalized Robot 3D Vision-Language Model with Fast Rendering and Pre-Training Vision-Language Alignment [55.11291053011696]
  This work presents a framework for dealing with 3D scene understanding when the labeled scenes are quite limited.<n>To extract knowledge for novel categories from the pre-trained vision-language models, we propose a hierarchical feature-aligned pre-training and knowledge distillation strategy.<n>In the limited reconstruction case, our proposed approach, termed WS3D++, ranks 1st on the large-scale ScanNet benchmark.
  arXiv  Detail & Related papers  (2023-12-01T15:47:04Z)
• ISIM: Iterative Self-Improved Model for Weakly Supervised Segmentation [0.34265828682659694]
  Weakly Supervised Semantic Conditional (WSSS) is a challenging task aiming to learn the segmentation labels from class-level labels. We propose a framework that employs an iterative approach in a modified encoder-decoder-based segmentation model. Experiments performed with DeepLabv3 and UNet models show a significant gain on the Pascal VOC12 dataset.
  arXiv  Detail & Related papers  (2022-11-22T18:14:06Z)
• LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
  We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning. Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
  arXiv  Detail & Related papers  (2022-10-14T19:13:36Z)
• Data Augmentation-free Unsupervised Learning for 3D Point Cloud Understanding [61.30276576646909]
  We propose an augmentation-free unsupervised approach for point clouds to learn transferable point-level features via soft clustering, named SoftClu. We exploit the affiliation of points to their clusters as a proxy to enable self-training through a pseudo-label prediction task.
  arXiv  Detail & Related papers  (2022-10-06T10:18:16Z)
• Improving Point Cloud Semantic Segmentation by Learning 3D Object Detection [102.62963605429508]
  Point cloud semantic segmentation plays an essential role in autonomous driving. Current 3D semantic segmentation networks focus on convolutional architectures that perform great for well represented classes. We propose a novel Aware 3D Semantic Detection (DASS) framework that explicitly leverages localization features from an auxiliary 3D object detection task.
  arXiv  Detail & Related papers  (2020-09-22T14:17:40Z)
• Fine-Grained Visual Classification with Efficient End-to-end Localization [49.9887676289364]
  We present an efficient localization module that can be fused with a classification network in an end-to-end setup. We evaluate the new model on the three benchmark datasets CUB200-2011, Stanford Cars and FGVC-Aircraft.
  arXiv  Detail & Related papers  (2020-05-11T14:07:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.