Trace3D: Consistent Segmentation Lifting via Gaussian Instance Tracing

• URL: http://arxiv.org/abs/2508.03227v1
• Date: Tue, 05 Aug 2025 08:54:17 GMT
• Title: Trace3D: Consistent Segmentation Lifting via Gaussian Instance Tracing
• Abstract summary: We address the challenge of lifting 2D visual segmentation to 3D in Gaussian Splatting.<n>Existing methods often suffer from inconsistent 2D masks across viewpoints and produce noisy segmentation boundaries.<n>We introduce Gaussian Instance Tracing (GIT), which augments the standard Gaussian representation with an instance weight matrix across input views.
• Score: 27.24794829116753
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We address the challenge of lifting 2D visual segmentation to 3D in Gaussian Splatting. Existing methods often suffer from inconsistent 2D masks across viewpoints and produce noisy segmentation boundaries as they neglect these semantic cues to refine the learned Gaussians. To overcome this, we introduce Gaussian Instance Tracing (GIT), which augments the standard Gaussian representation with an instance weight matrix across input views. Leveraging the inherent consistency of Gaussians in 3D, we use this matrix to identify and correct 2D segmentation inconsistencies. Furthermore, since each Gaussian ideally corresponds to a single object, we propose a GIT-guided adaptive density control mechanism to split and prune ambiguous Gaussians during training, resulting in sharper and more coherent 2D and 3D segmentation boundaries. Experimental results show that our method extracts clean 3D assets and consistently improves 3D segmentation in both online (e.g., self-prompting) and offline (e.g., contrastive lifting) settings, enabling applications such as hierarchical segmentation, object extraction, and scene editing.

Related papers

• PointGS: Semantic-Consistent Unsupervised 3D Point Cloud Segmentation with 3D Gaussian Splatting [15.208505879476418]
  Unsupervised point cloud segmentation is critical for embodied artificial intelligence and autonomous driving.<n>This paper proposes PointGS, a simple yet effective pipeline for unsupervised 3D point cloud segmentation.<n> Experiments demonstrate that PointGS outperforms state-of-the-art unsupervised methods.
  arXiv  Detail & Related papers  (2026-05-12T04:44:50Z)
• NRGS: Neural Regularization for Robust 3D Semantic Gaussian Splatting [41.21645092733966]
  We propose a neural regularization method that refines the noisy 3D semantic field produced by lifting multi-view inconsistent 2D features.<n> Experiments show that our method enhances the accuracy of lifted semantics, providing an efficient and effective approach to robust 3D semantic Gaussian Splatting.
  arXiv  Detail & Related papers  (2026-04-24T10:57:05Z)
• GaussianTrimmer: Online Trimming Boundaries for 3DGS Segmentation [29.760211457554348]
  We propose an online boundary trimming method, GaussianTrimmer, for 3D scene representation.<n>Our method can improve the segmentation quality of existing 3D Gaussian segmentation methods as a plug-and-play method.
  arXiv  Detail & Related papers  (2026-01-19T03:03:54Z)
• Joint Semantic and Rendering Enhancements in 3D Gaussian Modeling with Anisotropic Local Encoding [86.55824709875598]
  We propose a joint enhancement framework for 3D semantic Gaussian modeling that synergizes both semantic and rendering branches.<n>Unlike conventional point cloud shape encoding, we introduce an anisotropic 3D Gaussian Chebyshev descriptor to capture fine-grained 3D shape details.<n>We employ a cross-scene knowledge transfer module to continuously update learned shape patterns, enabling faster convergence and robust representations.
  arXiv  Detail & Related papers  (2026-01-05T18:33:50Z)
• LabelGS: Label-Aware 3D Gaussian Splatting for 3D Scene Segmentation [56.4321049923868]
  3D Gaussian Splatting (3DGS) has emerged as a novel explicit representation for 3D scenes, offering both high-fidelity reconstruction and efficient rendering.<n>We propose Label-aware 3D Gaussian Splatting (LabelGS), a method that augments the Gaussian representation with object label.<n>LabelGS achieves a remarkable 22X speedup in training compared to Feature-3DGS, at a resolution of 1440X1080.
  arXiv  Detail & Related papers  (2025-08-27T09:07:38Z)
• SAGOnline: Segment Any Gaussians Online [17.33447710659887]
  3D Gaussian Splatting (3DGS) has emerged as a powerful paradigm for explicit 3D scene representation.<n>Current methods suffer from prohibitive computational costs, limited 3D spatial reasoning, and an inability to track multiple objects simultaneously.<n>We present Segment Any Gaussians Online (SAGOnline), a lightweight and zero-shot framework for real-time 3D segmentation in Gaussian scenes.
  arXiv  Detail & Related papers  (2025-08-11T17:38:50Z)
• COB-GS: Clear Object Boundaries in 3DGS Segmentation Based on Boundary-Adaptive Gaussian Splitting [67.03992455145325]
  3D segmentation based on 3D Gaussian Splatting (3DGS) struggles with accurately delineating object boundaries.<n>We introduce Clear Object Boundaries for 3DGS (COB-GS), which aims to improve segmentation accuracy.<n>For semantic guidance, we introduce a boundary-adaptive Gaussian splitting technique.<n>For the visual optimization, we rectify the degraded texture of the 3DGS scene.
  arXiv  Detail & Related papers  (2025-03-25T08:31:43Z)
• Bootstraping Clustering of Gaussians for View-consistent 3D Scene Understanding [59.51535163599723]
  FreeGS is an unsupervised semantic-embedded 3DGS framework that achieves view-consistent 3D scene understanding without the need for 2D labels.<n>FreeGS performs comparably to state-of-the-art methods while avoiding the complex data preprocessing workload.
  arXiv  Detail & Related papers  (2024-11-29T08:52:32Z)
• ShapeSplat: A Large-scale Dataset of Gaussian Splats and Their Self-Supervised Pretraining [104.34751911174196]
  We build a large-scale dataset of 3DGS using ShapeNet and ModelNet datasets. Our dataset ShapeSplat consists of 65K objects from 87 unique categories. We introduce textbftextitGaussian-MAE, which highlights the unique benefits of representation learning from Gaussian parameters.
  arXiv  Detail & Related papers  (2024-08-20T14:49:14Z)
• CLIP-GS: CLIP-Informed Gaussian Splatting for View-Consistent 3D Indoor Semantic Understanding [17.440124130814166]
  Exploiting 3D Gaussian Splatting (3DGS) with Contrastive Language-Image Pre-Training (CLIP) models for open-vocabulary 3D semantic understanding of indoor scenes has emerged as an attractive research focus.<n>We present CLIP-GS, efficiently achieving a coherent semantic understanding of 3D indoor scenes via the proposed Semantic Attribute Compactness (SAC) and 3D Coherent Regularization (3DCR)<n>Our method remarkably suppresses existing state-of-the-art approaches, achieving mIoU improvements of 21.20% and 13.05% on ScanNet and Replica datasets, respectively
  arXiv  Detail & Related papers  (2024-04-22T15:01:32Z)
• Semantic Gaussians: Open-Vocabulary Scene Understanding with 3D Gaussian Splatting [27.974762304763694]
  We introduce Semantic Gaussians, a novel open-vocabulary scene understanding approach based on 3D Gaussian Splatting. Unlike existing methods, we design a versatile projection approach that maps various 2D semantic features into a novel semantic component of 3D Gaussians. We build a 3D semantic network that directly predicts the semantic component from raw 3D Gaussians for fast inference.
  arXiv  Detail & Related papers  (2024-03-22T21:28:19Z)
• SAGD: Boundary-Enhanced Segment Anything in 3D Gaussian via Gaussian Decomposition [66.56357905500512]
  3D Gaussian Splatting has emerged as an alternative 3D representation for novel view synthesis.<n>We propose SAGD, a conceptually simple yet effective boundary-enhanced segmentation pipeline for 3D-GS.<n>Our approach achieves high-quality 3D segmentation without rough boundary issues, which can be easily applied to other scene editing tasks.
  arXiv  Detail & Related papers  (2024-01-31T14:19:03Z)
• 2D-Guided 3D Gaussian Segmentation [15.139488857163064]
  This paper introduces a 3D Gaussian segmentation method implemented with 2D segmentation as supervision. This approach uses input 2D segmentation maps to guide the learning of the added 3D Gaussian semantic information. Experiments show that our method can achieve comparable performances on mIOU and mAcc for multi-object segmentation.
  arXiv  Detail & Related papers  (2023-12-26T13:28:21Z)
• Segment Any 3D Gaussians [85.93694310363325]
  This paper presents SAGA, a highly efficient 3D promptable segmentation method based on 3D Gaussian Splatting (3D-GS)<n>Given 2D visual prompts as input, SAGA can segment the corresponding 3D target represented by 3D Gaussians within 4 ms.<n>We show that SAGA achieves real-time multi-granularity segmentation with quality comparable to state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-12-01T17:15:24Z)
• Gaussian Grouping: Segment and Edit Anything in 3D Scenes [65.49196142146292]
  We propose Gaussian Grouping, which extends Gaussian Splatting to jointly reconstruct and segment anything in open-world 3D scenes. Compared to the implicit NeRF representation, we show that the grouped 3D Gaussians can reconstruct, segment and edit anything in 3D with high visual quality, fine granularity and efficiency.
  arXiv  Detail & Related papers  (2023-12-01T17:09:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.