Segmentation-Driven Initialization for Sparse-view 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2509.11853v1
• Date: Mon, 15 Sep 2025 12:31:33 GMT
• Title: Segmentation-Driven Initialization for Sparse-view 3D Gaussian Splatting
• Authors: Yi-Hsin Li, Thomas Sikora, Sebastian Knorr, Måarten Sjöström,
• Abstract summary: 3D Gaussian Splatting (3DGS) has enabled real-time rendering with competitive quality.<n>Existing pipelines often rely on Structure-from-Motion (SfM) for camera pose estimation, an approach that struggles in genuinely sparse-view settings.<n>We propose a method that mitigates inefficiency by leveraging region-based segmentation to identify and retain only structurally significant regions.
• Score: 0.9251324073335035
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Sparse-view synthesis remains a challenging problem due to the difficulty of recovering accurate geometry and appearance from limited observations. While recent advances in 3D Gaussian Splatting (3DGS) have enabled real-time rendering with competitive quality, existing pipelines often rely on Structure-from-Motion (SfM) for camera pose estimation, an approach that struggles in genuinely sparse-view settings. Moreover, several SfM-free methods replace SfM with multi-view stereo (MVS) models, but generate massive numbers of 3D Gaussians by back-projecting every pixel into 3D space, leading to high memory costs. We propose Segmentation-Driven Initialization for Gaussian Splatting (SDI-GS), a method that mitigates inefficiency by leveraging region-based segmentation to identify and retain only structurally significant regions. This enables selective downsampling of the dense point cloud, preserving scene fidelity while substantially reducing Gaussian count. Experiments across diverse benchmarks show that SDI-GS reduces Gaussian count by up to 50% and achieves comparable or superior rendering quality in PSNR and SSIM, with only marginal degradation in LPIPS. It further enables faster training and lower memory footprint, advancing the practicality of 3DGS for constrained-view scenarios.

Related papers

• Initialize to Generalize: A Stronger Initialization Pipeline for Sparse-View 3DGS [31.824354716471294]
  Sparse-view 3D Gaussian Splatting (3DGS) often overfits to the training views, leading to artifacts like blurring in novel view rendering.<n>Prior work addresses it either by enhancing the point cloud from Structure-from-Motion (SfM) or by adding training-time constraints (regularization) to the 3DGS optimization.<n>We design frequency-aware SfM that improves low-texture coverage via low-frequency view augmentation and relaxed multi-view correspondences.
  arXiv  Detail & Related papers  (2025-10-20T12:23:19Z)
• Steepest Descent Density Control for Compact 3D Gaussian Splatting [72.54055499344052]
  3D Gaussian Splatting (3DGS) has emerged as a powerful real-time, high-resolution novel view.<n>We propose a theoretical framework that demystifies and improves density control in 3DGS.<n>We introduce SteepGS, incorporating steepest density control, a principled strategy that minimizes loss while maintaining a compact point cloud.
  arXiv  Detail & Related papers  (2025-05-08T18:41:38Z)
• PG-SAG: Parallel Gaussian Splatting for Fine-Grained Large-Scale Urban Buildings Reconstruction via Semantic-Aware Grouping [6.160345720038265]
  We introduce a parallel Gaussian splatting method, termed PG-SAG, which fully exploits semantic cues for both partitioning and kernel optimization.<n>Experiments are tested on various urban datasets, the results demonstrated the superior performance of our PG-SAG on building surface reconstruction.
  arXiv  Detail & Related papers  (2025-01-03T07:40:16Z)
• MCGS: Multiview Consistency Enhancement for Sparse-View 3D Gaussian Radiance Fields [100.90743697473232]
  Radiance fields represented by 3D Gaussians excel at synthesizing novel views, offering both high training efficiency and fast rendering.<n>Existing methods often incorporate depth priors from dense estimation networks but overlook the inherent multi-view consistency in input images.<n>We propose a view synthesis framework based on 3D Gaussian Splatting, enabling scene reconstruction from sparse views.
  arXiv  Detail & Related papers  (2024-10-15T08:39:05Z)
• LoopSparseGS: Loop Based Sparse-View Friendly Gaussian Splatting [18.682864169561498]
  LoopSparseGS is a loop-based 3DGS framework for the sparse novel view synthesis task. We introduce a novel Sparse-friendly Sampling (SFS) strategy to handle oversized Gaussian ellipsoids leading to large pixel errors. Experiments on four datasets demonstrate that LoopSparseGS outperforms existing state-of-the-art methods for sparse-input novel view synthesis.
  arXiv  Detail & Related papers  (2024-08-01T03:26:50Z)
• InstantSplat: Sparse-view Gaussian Splatting in Seconds [91.77050739918037]
  We introduce InstantSplat, a novel approach for addressing sparse-view 3D scene reconstruction at lightning-fast speed.<n>InstantSplat employs a self-supervised framework that optimize 3D scene representation and camera poses.<n>It achieves an acceleration of over 30x in reconstruction and improves visual quality (SSIM) from 0.3755 to 0.7624 compared to traditional SfM with 3D-GS.
  arXiv  Detail & Related papers  (2024-03-29T17:29:58Z)
• Relaxing Accurate Initialization Constraint for 3D Gaussian Splatting [29.58220473268378]
  We propose a novel optimization strategy dubbed RAIN-GS (Relaing Accurate Initialization Constraint for 3D Gaussian Splatting) RAIN-GS successfully trains 3D Gaussians from sub-optimal point cloud (e.g., randomly point cloud) We demonstrate the efficacy of our strategy through quantitative and qualitative comparisons on multiple datasets, where RAIN-GS trained with random point cloud achieves performance on-par with or even better than 3DGS trained with accurate SfM point cloud.
  arXiv  Detail & Related papers  (2024-03-14T14:04:21Z)
• GaussianPro: 3D Gaussian Splatting with Progressive Propagation [49.918797726059545]
  3DGS relies heavily on the point cloud produced by Structure-from-Motion (SfM) techniques. We propose a novel method that applies a progressive propagation strategy to guide the densification of the 3D Gaussians. Our method significantly surpasses 3DGS on the dataset, exhibiting an improvement of 1.15dB in terms of PSNR.
  arXiv  Detail & Related papers  (2024-02-22T16:00:20Z)
• GES: Generalized Exponential Splatting for Efficient Radiance Field Rendering [112.16239342037714]
  GES (Generalized Exponential Splatting) is a novel representation that employs Generalized Exponential Function (GEF) to model 3D scenes. With the aid of a frequency-modulated loss, GES achieves competitive performance in novel-view synthesis benchmarks.
  arXiv  Detail & Related papers  (2024-02-15T17:32:50Z)
• FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting [58.41056963451056]
  We propose a few-shot view synthesis framework based on 3D Gaussian Splatting. This framework enables real-time and photo-realistic view synthesis with as few as three training views. FSGS achieves state-of-the-art performance in both accuracy and rendering efficiency across diverse datasets.
  arXiv  Detail & Related papers  (2023-12-01T09:30:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.