2D-Guided 3D Gaussian Segmentation
- URL: http://arxiv.org/abs/2312.16047v1
- Date: Tue, 26 Dec 2023 13:28:21 GMT
- Title: 2D-Guided 3D Gaussian Segmentation
- Authors: Kun Lan, Haoran Li, Haolin Shi, Wenjun Wu, Yong Liao, Lin Wang,
Pengyuan Zhou
- Abstract summary: This paper introduces a 3D Gaussian segmentation method implemented with 2D segmentation as supervision.
This approach uses input 2D segmentation maps to guide the learning of the added 3D Gaussian semantic information.
Experiments show that our method can achieve comparable performances on mIOU and mAcc for multi-object segmentation.
- Score: 15.139488857163064
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Recently, 3D Gaussian, as an explicit 3D representation method, has
demonstrated strong competitiveness over NeRF (Neural Radiance Fields) in terms
of expressing complex scenes and training duration. These advantages signal a
wide range of applications for 3D Gaussians in 3D understanding and editing.
Meanwhile, the segmentation of 3D Gaussians is still in its infancy. The
existing segmentation methods are not only cumbersome but also incapable of
segmenting multiple objects simultaneously in a short amount of time. In
response, this paper introduces a 3D Gaussian segmentation method implemented
with 2D segmentation as supervision. This approach uses input 2D segmentation
maps to guide the learning of the added 3D Gaussian semantic information, while
nearest neighbor clustering and statistical filtering refine the segmentation
results. Experiments show that our concise method can achieve comparable
performances on mIOU and mAcc for multi-object segmentation as previous
single-object segmentation methods.
Related papers
- Click-Gaussian: Interactive Segmentation to Any 3D Gaussians [2.8461293457421957]
We propose Click-Gaussian, which learns distinguishable feature fields of two-level granularity.
Our method runs in 10 ms per click, 15 to 130 times as fast as the previous methods.
arXiv Detail & Related papers (2024-07-16T14:49:27Z) - View-Consistent Hierarchical 3D Segmentation Using Ultrametric Feature Fields [52.08335264414515]
We learn a novel feature field within a Neural Radiance Field (NeRF) representing a 3D scene.
Our method takes view-inconsistent multi-granularity 2D segmentations as input and produces a hierarchy of 3D-consistent segmentations as output.
We evaluate our method and several baselines on synthetic datasets with multi-view images and multi-granular segmentation, showcasing improved accuracy and viewpoint-consistency.
arXiv Detail & Related papers (2024-05-30T04:14:58Z) - SAGD: Boundary-Enhanced Segment Anything in 3D Gaussian via Gaussian Decomposition [66.80822249039235]
3D Gaussian Splatting has emerged as an alternative 3D representation for novel view synthesis.
We propose SAGD, a conceptually simple yet effective boundary-enhanced segmentation pipeline for 3D-GS.
Our approach achieves high-quality 3D segmentation without rough boundary issues, which can be easily applied to other scene editing tasks.
arXiv Detail & Related papers (2024-01-31T14:19:03Z) - Segment3D: Learning Fine-Grained Class-Agnostic 3D Segmentation without
Manual Labels [141.23836433191624]
Current 3D scene segmentation methods are heavily dependent on manually annotated 3D training datasets.
We propose Segment3D, a method for class-agnostic 3D scene segmentation that produces high-quality 3D segmentation masks.
arXiv Detail & Related papers (2023-12-28T18:57:11Z) - SAM-guided Graph Cut for 3D Instance Segmentation [63.797612618531346]
This paper addresses the challenge of 3D instance segmentation by simultaneously leveraging 3D geometric and multi-view image information.
We introduce a novel 3D-to-2D query framework to effectively exploit 2D segmentation models for 3D instance segmentation.
Our method achieves robust segmentation performance and can generalize across different types of scenes.
arXiv Detail & Related papers (2023-12-13T18:59:58Z) - PartSLIP++: Enhancing Low-Shot 3D Part Segmentation via Multi-View
Instance Segmentation and Maximum Likelihood Estimation [32.2861030554128]
PartSLIP, a recent advancement, has made significant strides in zero- and few-shot 3D part segmentation.
We introduce PartSLIP++, an enhanced version designed to overcome the limitations of its predecessor.
We show that PartSLIP++ demonstrates better performance over PartSLIP in both low-shot 3D semantic and instance-based object part segmentation tasks.
arXiv Detail & Related papers (2023-12-05T01:33:04Z) - Segment Any 3D Gaussians [85.93694310363325]
This paper presents SAGA, a highly efficient 3D promptable segmentation method based on 3D Gaussian Splatting (3D-GS)
Given 2D visual prompts as input, SAGA can segment the corresponding 3D target represented by 3D Gaussians within 4 ms.
We show that SAGA achieves real-time multi-granularity segmentation with quality comparable to state-of-the-art methods.
arXiv Detail & Related papers (2023-12-01T17:15:24Z) - Gaussian Grouping: Segment and Edit Anything in 3D Scenes [65.49196142146292]
We propose Gaussian Grouping, which extends Gaussian Splatting to jointly reconstruct and segment anything in open-world 3D scenes.
Compared to the implicit NeRF representation, we show that the grouped 3D Gaussians can reconstruct, segment and edit anything in 3D with high visual quality, fine granularity and efficiency.
arXiv Detail & Related papers (2023-12-01T17:09:31Z) - Adjacent Slice Feature Guided 2.5D Network for Pulmonary Nodule
Segmentation [11.960631781470811]
2D segmentation methods with less parameters and calculation have the problem of lacking spatial relations between slices.
In this paper, we propose an adjacent slice feature guided 2.5D network to solve this problem.
Our method performs better than other existing methods in pulmonary nodule segmentation task.
arXiv Detail & Related papers (2022-11-19T06:13:18Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.