Initialize to Generalize: A Stronger Initialization Pipeline for Sparse-View 3DGS

• URL: http://arxiv.org/abs/2510.17479v1
• Date: Mon, 20 Oct 2025 12:23:19 GMT
• Title: Initialize to Generalize: A Stronger Initialization Pipeline for Sparse-View 3DGS
• Authors: Feng Zhou, Wenkai Guo, Pu Cao, Zhicheng Zhang, Jianqin Yin,
• Abstract summary: Sparse-view 3D Gaussian Splatting (3DGS) often overfits to the training views, leading to artifacts like blurring in novel view rendering.<n>Prior work addresses it either by enhancing the point cloud from Structure-from-Motion (SfM) or by adding training-time constraints (regularization) to the 3DGS optimization.<n>We design frequency-aware SfM that improves low-texture coverage via low-frequency view augmentation and relaxed multi-view correspondences.
• Score: 31.824354716471294
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sparse-view 3D Gaussian Splatting (3DGS) often overfits to the training views, leading to artifacts like blurring in novel view rendering. Prior work addresses it either by enhancing the initialization (\emph{i.e.}, the point cloud from Structure-from-Motion (SfM)) or by adding training-time constraints (regularization) to the 3DGS optimization. Yet our controlled ablations reveal that initialization is the decisive factor: it determines the attainable performance band in sparse-view 3DGS, while training-time constraints yield only modest within-band improvements at extra cost. Given initialization's primacy, we focus our design there. Although SfM performs poorly under sparse views due to its reliance on feature matching, it still provides reliable seed points. Thus, building on SfM, our effort aims to supplement the regions it fails to cover as comprehensively as possible. Specifically, we design: (i) frequency-aware SfM that improves low-texture coverage via low-frequency view augmentation and relaxed multi-view correspondences; (ii) 3DGS self-initialization that lifts photometric supervision into additional points, compensating SfM-sparse regions with learned Gaussian centers; and (iii) point-cloud regularization that enforces multi-view consistency and uniform spatial coverage through simple geometric/visibility priors, yielding a clean and reliable point cloud. Our experiments on LLFF and Mip-NeRF360 demonstrate consistent gains in sparse-view settings, establishing our approach as a stronger initialization strategy. Code is available at https://github.com/zss171999645/ItG-GS.

Related papers

• Global-Aware Edge Prioritization for Pose Graph Initialization [58.77851776918465]
  A core component of Structure-from-Motion (SfM) where images act as nodes and edges encode relative poses.<n>Since geometric verification is expensive, SfM pipelines restrict the pose graph to a sparse set of candidate edges.<n>We address this limitation through the concept of edge prioritization, ranking candidate edges by their utility for SfM.<n>Our approach has three components: (1) a GNN trained with SfM-derived supervision to predict globally consistent edge reliability; (2) multi-minimal-spanning-tree-based pose graph construction guided by these ranks; and (3) connectivity-aware score modulation that reinforces weak regions
  arXiv  Detail & Related papers  (2026-02-25T14:44:53Z)
• Pi-GS: Sparse-View Gaussian Splatting with Dense π^3 Initialization [5.5775900281150514]
  We propose a robust method utilizing 3, a reference-free point cloud estimation network.<n>We employ uncertainty-guided depth supervision, normal consistency loss, and depth warping.<n>Our approach achieves state-of-the-art performance on the Tanks and Temples, LLFF, DTU, and MipNeRF360 datasets.
  arXiv  Detail & Related papers  (2026-02-03T09:55:03Z)
• Segmentation-Driven Initialization for Sparse-view 3D Gaussian Splatting [0.9251324073335035]
  3D Gaussian Splatting (3DGS) has enabled real-time rendering with competitive quality.<n>Existing pipelines often rely on Structure-from-Motion (SfM) for camera pose estimation, an approach that struggles in genuinely sparse-view settings.<n>We propose a method that mitigates inefficiency by leveraging region-based segmentation to identify and retain only structurally significant regions.
  arXiv  Detail & Related papers  (2025-09-15T12:31:33Z)
• AttentionGS: Towards Initialization-Free 3D Gaussian Splatting via Structural Attention [11.400892739301804]
  3D Gaussian Splatting (3DGS) is a powerful alternative to Neural Radiance Fields (NeRF)<n>It relies on high-quality point clouds from Structure-from-Motion (SfM)<n>We propose AttentionGS, a novel framework that eliminates the dependency on high-quality initial point clouds.
  arXiv  Detail & Related papers  (2025-06-30T08:16:43Z)
• Intern-GS: Vision Model Guided Sparse-View 3D Gaussian Splatting [95.61137026932062]
  Intern-GS is a novel approach to enhance the process of sparse-view Gaussian splatting.<n>We show that Intern-GS achieves state-of-the-art rendering quality across diverse datasets.
  arXiv  Detail & Related papers  (2025-05-27T05:17:49Z)
• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting [54.7468067660037]
  PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices.<n>Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS.
  arXiv  Detail & Related papers  (2024-10-29T15:28:15Z)
• MCGS: Multiview Consistency Enhancement for Sparse-View 3D Gaussian Radiance Fields [100.90743697473232]
  Radiance fields represented by 3D Gaussians excel at synthesizing novel views, offering both high training efficiency and fast rendering.<n>Existing methods often incorporate depth priors from dense estimation networks but overlook the inherent multi-view consistency in input images.<n>We propose a view synthesis framework based on 3D Gaussian Splatting, enabling scene reconstruction from sparse views.
  arXiv  Detail & Related papers  (2024-10-15T08:39:05Z)
• Dense Point Clouds Matter: Dust-GS for Scene Reconstruction from Sparse Viewpoints [9.069919085326]
  3D Gaussian Splatting (3DGS) has demonstrated remarkable performance in scene synthesis and novel view synthesis tasks. In this study, we present Dust-GS, a novel framework specifically designed to overcome the limitations of 3DGS in sparse viewpoint conditions.
  arXiv  Detail & Related papers  (2024-09-13T07:59:15Z)
• InstantSplat: Sparse-view Gaussian Splatting in Seconds [91.77050739918037]
  We introduce InstantSplat, a novel approach for addressing sparse-view 3D scene reconstruction at lightning-fast speed.<n>InstantSplat employs a self-supervised framework that optimize 3D scene representation and camera poses.<n>It achieves an acceleration of over 30x in reconstruction and improves visual quality (SSIM) from 0.3755 to 0.7624 compared to traditional SfM with 3D-GS.
  arXiv  Detail & Related papers  (2024-03-29T17:29:58Z)
• Relaxing Accurate Initialization Constraint for 3D Gaussian Splatting [29.58220473268378]
  We propose a novel optimization strategy dubbed RAIN-GS (Relaing Accurate Initialization Constraint for 3D Gaussian Splatting) RAIN-GS successfully trains 3D Gaussians from sub-optimal point cloud (e.g., randomly point cloud) We demonstrate the efficacy of our strategy through quantitative and qualitative comparisons on multiple datasets, where RAIN-GS trained with random point cloud achieves performance on-par with or even better than 3DGS trained with accurate SfM point cloud.
  arXiv  Detail & Related papers  (2024-03-14T14:04:21Z)
• FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting [58.41056963451056]
  We propose a few-shot view synthesis framework based on 3D Gaussian Splatting. This framework enables real-time and photo-realistic view synthesis with as few as three training views. FSGS achieves state-of-the-art performance in both accuracy and rendering efficiency across diverse datasets.
  arXiv  Detail & Related papers  (2023-12-01T09:30:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.