Deep Learning Based 3D Segmentation: A Survey
- URL: http://arxiv.org/abs/2103.05423v3
- Date: Wed, 26 Jul 2023 08:14:39 GMT
- Title: Deep Learning Based 3D Segmentation: A Survey
- Authors: Yong He, Hongshan Yu, Xiaoyan Liu, Zhengeng Yang, Wei Sun, Ajmal Mian
- Abstract summary: 3D segmentation is a fundamental problem in computer vision with applications in autonomous driving, robotics, augmented reality and medical image analysis.
Deep learning techniques have recently become the tool of choice for 3D segmentation tasks.
This paper fills the gap and provides a comprehensive survey of the recent progress made in deep learning based 3D segmentation.
- Score: 29.402585297221457
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: 3D segmentation is a fundamental and challenging problem in computer vision
with applications in autonomous driving, robotics, augmented reality and
medical image analysis. It has received significant attention from the computer
vision, graphics and machine learning communities. Conventional methods for 3D
segmentation, based on hand-crafted features and machine learning classifiers,
lack generalization ability. Driven by their success in 2D computer vision,
deep learning techniques have recently become the tool of choice for 3D
segmentation tasks. This has led to an influx of a large number of methods in
the literature that have been evaluated on different benchmark datasets.
Whereas survey papers on RGB-D and point cloud segmentation exist, there is a
lack of an in-depth and recent survey that covers all 3D data modalities and
application domains. This paper fills the gap and provides a comprehensive
survey of the recent progress made in deep learning based 3D segmentation. It
covers over 180 works, analyzes their strengths and limitations and discusses
their competitive results on benchmark datasets. The survey provides a summary
of the most commonly used pipelines and finally highlights promising research
directions for the future.
Related papers
- 3D Instance Segmentation Using Deep Learning on RGB-D Indoor Data [0.0]
2D region based convolutional neural networks (Mask R-CNN) deep learning model with point based rending module is adapted to integrate with depth information to recognize and segment 3D instances of objects.
In order to generate 3D point cloud coordinates, segmented 2D pixels of recognized object regions in the RGB image are merged into (u, v) points of the depth image.
arXiv Detail & Related papers (2024-06-19T08:00:35Z) - Deep Learning-Based 3D Instance and Semantic Segmentation: A Review [0.0]
3D segmentation is challenging with point cloud data due to substantial redundancy, fluctuating sample density and lack of organization.
Deep learning has been successfully used to a spectrum of 2D vision domains as a prevailing A.I. methods.
This study examines many strategies that have been presented to 3D instance and semantic segmentation.
arXiv Detail & Related papers (2024-06-19T07:56:14Z) - Reasoning3D -- Grounding and Reasoning in 3D: Fine-Grained Zero-Shot Open-Vocabulary 3D Reasoning Part Segmentation via Large Vision-Language Models [20.277479473218513]
We introduce a new task: Zero-Shot 3D Reasoning for parts searching and localization for objects.
We design a simple baseline method, Reasoning3D, with the capability to understand and execute complex commands.
We show that Reasoning3D can effectively localize and highlight parts of 3D objects based on implicit textual queries.
arXiv Detail & Related papers (2024-05-29T17:56:07Z) - Reason3D: Searching and Reasoning 3D Segmentation via Large Language Model [108.35777542298224]
This paper introduces Reason3D, a novel large language model for comprehensive 3D understanding.
We propose a hierarchical mask decoder to locate small objects within expansive scenes.
Experiments validate that Reason3D achieves remarkable results on large-scale ScanNet and Matterport3D datasets.
arXiv Detail & Related papers (2024-05-27T17:59:41Z) - A comprehensive overview of deep learning techniques for 3D point cloud classification and semantic segmentation [0.20649496811699863]
This paper analyzes recent progress in deep learning methods employed for point cloud processing.
It serves as a comprehensive review on two major tasks in 3D point cloud processing-- namely, 3D shape classification and semantic segmentation.
arXiv Detail & Related papers (2024-05-20T09:33:27Z) - Leveraging Large-Scale Pretrained Vision Foundation Models for
Label-Efficient 3D Point Cloud Segmentation [67.07112533415116]
We present a novel framework that adapts various foundational models for the 3D point cloud segmentation task.
Our approach involves making initial predictions of 2D semantic masks using different large vision models.
To generate robust 3D semantic pseudo labels, we introduce a semantic label fusion strategy that effectively combines all the results via voting.
arXiv Detail & Related papers (2023-11-03T15:41:15Z) - A Threefold Review on Deep Semantic Segmentation: Efficiency-oriented,
Temporal and Depth-aware design [77.34726150561087]
We conduct a survey on the most relevant and recent advances in Deep Semantic in the context of vision for autonomous vehicles.
Our main objective is to provide a comprehensive discussion on the main methods, advantages, limitations, results and challenges faced from each perspective.
arXiv Detail & Related papers (2023-03-08T01:29:55Z) - PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained
Image-Language Models [56.324516906160234]
Generalizable 3D part segmentation is important but challenging in vision and robotics.
This paper explores an alternative way for low-shot part segmentation of 3D point clouds by leveraging a pretrained image-language model, GLIP.
We transfer the rich knowledge from 2D to 3D through GLIP-based part detection on point cloud rendering and a novel 2D-to-3D label lifting algorithm.
arXiv Detail & Related papers (2022-12-03T06:59:01Z) - Improving Point Cloud Semantic Segmentation by Learning 3D Object
Detection [102.62963605429508]
Point cloud semantic segmentation plays an essential role in autonomous driving.
Current 3D semantic segmentation networks focus on convolutional architectures that perform great for well represented classes.
We propose a novel Aware 3D Semantic Detection (DASS) framework that explicitly leverages localization features from an auxiliary 3D object detection task.
arXiv Detail & Related papers (2020-09-22T14:17:40Z) - Deep Learning for 3D Point Cloud Understanding: A Survey [16.35767262996978]
The development of practical applications, such as autonomous driving and robotics, has brought increasing attention to 3D point cloud understanding.
Deep learning has achieved remarkable success on image-based tasks, but there are many unique challenges faced by deep neural networks in processing massive, unstructured and noisy 3D points.
This paper summarizes recent remarkable research contributions in this area from several different directions.
arXiv Detail & Related papers (2020-09-18T16:34:12Z) - PointContrast: Unsupervised Pre-training for 3D Point Cloud
Understanding [107.02479689909164]
In this work, we aim at facilitating research on 3D representation learning.
We measure the effect of unsupervised pre-training on a large source set of 3D scenes.
arXiv Detail & Related papers (2020-07-21T17:59:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.