Robust Prior-Guided Segmentation for Editable 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2605.16065v1
• Date: Fri, 15 May 2026 15:29:30 GMT
• Title: Robust Prior-Guided Segmentation for Editable 3D Gaussian Splatting
• Abstract summary: Existing approaches that lift 2D segmentations to the 3D domain suffer from view inconsistencies and coarse masks.<n>We propose a novel framework that leverages the Segment Anything Model High Quality to generate accurate 2D masks.<n>Our approach achieves state-of-the-art segmentation accuracy and enables interactive, real-time object editing while maintaining high visual fidelity.
• Score: 6.290910602456353
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D Gaussian Splatting (3D-GS) enables real-time 3D scene reconstruction but lacks robust segmentation for editing tasks such as object removal, extraction, and recoloring. Existing approaches that lift 2D segmentations to the 3D domain suffer from view inconsistencies and coarse masks. In this paper, we propose a novel framework that leverages the Segment Anything Model High Quality (SAM-HQ) to generate accurate 2D masks, addressing the limitations of the standard SAM in boundary fidelity and fine-structure preservation. To achieve robust 3D segmentation of any target object in a given scene, we introduce a prior-guided label reassignment method that assigns labels to 3D Gaussians by enforcing multiview consistency with learned priors. Our approach achieves state-of-the-art segmentation accuracy and enables interactive, real-time object editing while maintaining high visual fidelity. Qualitative results demonstrate superior boundary preservation and practical utility in Virtual Reality (VR) and robotics, advancing 3D scene editing.

Related papers

• CoIn: Comprehensive 2D-3D Inpainting with Gaussian Splatting Guidance [33.42204651464501]
  CoIn is a framework that bridges 2D inpainting models and 3DGS through a multi-stage consistency pipeline.<n>We introduce Reference Adaptive GS with Feature Attention to reconstruct a coarse 3D scene by adaptively weighing towards a reference view (2D -> 3D)<n>Finally, a Texture-Enhancing Discriminator refines the 3D scene to achieve high photometric realism (2D -> 3D)
  arXiv  Detail & Related papers  (2026-06-25T22:24:27Z)
• VGGT-Edit: Feed-forward Native 3D Scene Editing with Residual Field Prediction [59.303842406260124]
  VGGT-Edit is a feed-forward framework for text-conditioned native 3D scene editing.<n>We introduce depth-synchronized text injection to align semantic guidance with the backbone's spatial poses.<n> VGGT-Edit substantially outperforms 2D-lifting baselines, producing sharper object details, stronger multi-view consistency, and near-instant inference speed.
  arXiv  Detail & Related papers  (2026-05-14T17:59:04Z)
• Class-agnostic 3D Segmentation by Granularity-Consistent Automatic 2D Mask Tracking [10.223105883919278]
  We introduce a Granularity-Consistent automatic 2D Mask Tracking approach that maintains temporal correspondences across frames.<n>Our method effectively generated consistent and accurate 3D segmentations.
  arXiv  Detail & Related papers  (2025-11-02T03:52:42Z)
• SGS-3D: High-Fidelity 3D Instance Segmentation via Reliable Semantic Mask Splitting and Growing [20.383892902000976]
  We propose splitting and growing reliable semantic masks for high-fidelity 3D instance segmentation (SGS-3D)<n>For semantic guidance, we introduce a mask filtering strategy that leverages the co-occurrence of 3D geometry primitives.<n>For the geometric refinement, we construct fine-grained object instances by exploiting both spatial continuity and high-level features.
  arXiv  Detail & Related papers  (2025-09-05T14:37:31Z)
• Integrating SAM Supervision for 3D Weakly Supervised Point Cloud Segmentation [66.65719382619538]
  Current methods for 3D semantic segmentation propose training models with limited annotations to address the difficulty of annotating large, irregular, and unordered 3D point cloud data.<n>We present a novel approach that maximizes the utility of sparsely available 3D annotations incorporating segmentation masks generated by 2D foundation models.
  arXiv  Detail & Related papers  (2025-08-27T14:13:01Z)
• GeoSAM2: Unleashing the Power of SAM2 for 3D Part Segmentation [81.0871900167463]
  We introduce GeoSAM2, a prompt-controllable framework for 3D part segmentation.<n>Given a textureless object, we render normal and point maps from predefined viewpoints.<n>We accept simple 2D prompts - clicks or boxes - to guide part selection.<n>The predicted masks are back-projected to the object and aggregated across views.
  arXiv  Detail & Related papers  (2025-08-19T17:58:51Z)
• SAGOnline: Segment Any Gaussians Online [17.33447710659887]
  3D Gaussian Splatting (3DGS) has emerged as a powerful paradigm for explicit 3D scene representation.<n>Current methods suffer from prohibitive computational costs, limited 3D spatial reasoning, and an inability to track multiple objects simultaneously.<n>We present Segment Any Gaussians Online (SAGOnline), a lightweight and zero-shot framework for real-time 3D segmentation in Gaussian scenes.
  arXiv  Detail & Related papers  (2025-08-11T17:38:50Z)
• SeqAffordSplat: Scene-level Sequential Affordance Reasoning on 3D Gaussian Splatting [85.87902260102652]
  We introduce the novel task of Sequential 3D Gaussian Affordance Reasoning.<n>We then propose SeqSplatNet, an end-to-end framework that directly maps an instruction to a sequence of 3D affordance masks.<n>Our method sets a new state-of-the-art on our challenging benchmark, effectively advancing affordance reasoning from single-step interactions to complex, sequential tasks at the scene level.
  arXiv  Detail & Related papers  (2025-07-31T17:56:55Z)
• Robust 3D-Masked Part-level Editing in 3D Gaussian Splatting with Regularized Score Distillation Sampling [13.107571213220123]
  RoMaP is a novel 3D Gaussian editing framework that enables precise and drastic part-level modifications.<n>We introduce a robust 3D mask generation module with our 3D-Geometry Label Aware Prediction.<n>We also propose a regularized SDS loss that combines the standard SDS loss with additional regularizers.
  arXiv  Detail & Related papers  (2025-07-15T07:54:11Z)
• T-3DGS: Removing Transient Objects for 3D Scene Reconstruction [83.05271859398779]
  Transient objects in video sequences can significantly degrade the quality of 3D scene reconstructions.<n>We propose T-3DGS, a novel framework that robustly filters out transient distractors during 3D reconstruction using Gaussian Splatting.
  arXiv  Detail & Related papers  (2024-11-29T07:45:24Z)
• Memorize What Matters: Emergent Scene Decomposition from Multitraverse [54.487589469432706]
  We introduce 3D Gaussian Mapping, a camera-only offline mapping framework grounded in 3D Gaussian Splatting. 3DGM converts multitraverse RGB videos from the same region into a Gaussian-based environmental map while concurrently performing 2D ephemeral object segmentation. We build the Mapverse benchmark, sourced from the Ithaca365 and nuPlan datasets, to evaluate our method in unsupervised 2D segmentation, 3D reconstruction, and neural rendering.
  arXiv  Detail & Related papers  (2024-05-27T14:11:17Z)
• SAGD: Boundary-Enhanced Segment Anything in 3D Gaussian via Gaussian Decomposition [66.56357905500512]
  3D Gaussian Splatting has emerged as an alternative 3D representation for novel view synthesis.<n>We propose SAGD, a conceptually simple yet effective boundary-enhanced segmentation pipeline for 3D-GS.<n>Our approach achieves high-quality 3D segmentation without rough boundary issues, which can be easily applied to other scene editing tasks.
  arXiv  Detail & Related papers  (2024-01-31T14:19:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.