Intern-GS: Vision Model Guided Sparse-View 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2505.20729v1
• Date: Tue, 27 May 2025 05:17:49 GMT
• Title: Intern-GS: Vision Model Guided Sparse-View 3D Gaussian Splatting
• Authors: Xiangyu Sun, Runnan Chen, Mingming Gong, Dong Xu, Tongliang Liu,
• Abstract summary: Intern-GS is a novel approach to enhance the process of sparse-view Gaussian splatting.<n>We show that Intern-GS achieves state-of-the-art rendering quality across diverse datasets.
• Score: 95.61137026932062
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sparse-view scene reconstruction often faces significant challenges due to the constraints imposed by limited observational data. These limitations result in incomplete information, leading to suboptimal reconstructions using existing methodologies. To address this, we present Intern-GS, a novel approach that effectively leverages rich prior knowledge from vision foundation models to enhance the process of sparse-view Gaussian Splatting, thereby enabling high-quality scene reconstruction. Specifically, Intern-GS utilizes vision foundation models to guide both the initialization and the optimization process of 3D Gaussian splatting, effectively addressing the limitations of sparse inputs. In the initialization process, our method employs DUSt3R to generate a dense and non-redundant gaussian point cloud. This approach significantly alleviates the limitations encountered by traditional structure-from-motion (SfM) methods, which often struggle under sparse-view constraints. During the optimization process, vision foundation models predict depth and appearance for unobserved views, refining the 3D Gaussians to compensate for missing information in unseen regions. Extensive experiments demonstrate that Intern-GS achieves state-of-the-art rendering quality across diverse datasets, including both forward-facing and large-scale scenes, such as LLFF, DTU, and Tanks and Temples.

Related papers

• QuickSplat: Fast 3D Surface Reconstruction via Learned Gaussian Initialization [69.50126552763157]
  Surface reconstruction is fundamental to computer vision and graphics, enabling applications in 3D modeling, mixed reality, robotics, and more.<n>Existing approaches based on rendering obtain promising results, but optimize on a per-scene basis, resulting in a slow optimization that can struggle to model textureless regions.<n>We introduce QuickSplat, which learns data-driven priors to generate dense initializations for 2D gaussian splatting optimization of large-scale indoor scenes.
  arXiv  Detail & Related papers  (2025-05-08T18:43:26Z)
• Diffusion-Guided Gaussian Splatting for Large-Scale Unconstrained 3D Reconstruction and Novel View Synthesis [22.767866875051013]
  We propose GS-Diff, a novel 3DGS framework guided by a multi-view diffusion model to address limitations of current methods.<n>By generating pseudo-observations conditioned on multi-view inputs, our method transforms under-constrained 3D reconstruction problems into well-posed ones.<n> Experiments on four benchmarks demonstrate that GS-Diff consistently outperforms state-of-the-art baselines by significant margins.
  arXiv  Detail & Related papers  (2025-04-02T17:59:46Z)
• EVolSplat: Efficient Volume-based Gaussian Splatting for Urban View Synthesis [61.1662426227688]
  Existing NeRF and 3DGS-based methods show promising results in achieving photorealistic renderings but require slow, per-scene optimization.<n>We introduce EVolSplat, an efficient 3D Gaussian Splatting model for urban scenes that works in a feed-forward manner.
  arXiv  Detail & Related papers  (2025-03-26T02:47:27Z)
• Taming Video Diffusion Prior with Scene-Grounding Guidance for 3D Gaussian Splatting from Sparse Inputs [28.381287866505637]
  We propose a reconstruction by generation pipeline that leverages learned priors from video diffusion models to provide plausible interpretations for regions outside the field of view or occluded.<n>We introduce a novel scene-grounding guidance based on rendered sequences from an optimized 3DGS, which tames the diffusion model to generate consistent sequences.<n>Our method significantly improves upon the baseline and achieves state-of-the-art performance on challenging benchmarks.
  arXiv  Detail & Related papers  (2025-03-07T01:59:05Z)
• See In Detail: Enhancing Sparse-view 3D Gaussian Splatting with Local Depth and Semantic Regularization [14.239772421978373]
  3D Gaussian Splatting (3DGS) has shown remarkable performance in novel view synthesis.<n>However, its rendering quality deteriorates with sparse inphut views, leading to distorted content and reduced details.<n>We propose a sparse-view 3DGS method, incorporating prior information is crucial.<n>Our method outperforms state-of-the-art novel view synthesis approaches, achieving up to 0.4dB improvement in terms of PSNR on the LLFF dataset.
  arXiv  Detail & Related papers  (2025-01-20T14:30:38Z)
• MonoGSDF: Exploring Monocular Geometric Cues for Gaussian Splatting-Guided Implicit Surface Reconstruction [84.07233691641193]
  We introduce MonoGSDF, a novel method that couples primitives with a neural Signed Distance Field (SDF) for high-quality reconstruction.<n>To handle arbitrary-scale scenes, we propose a scaling strategy for robust generalization.<n>Experiments on real-world datasets outperforms prior methods while maintaining efficiency.
  arXiv  Detail & Related papers  (2024-11-25T20:07:07Z)
• MCGS: Multiview Consistency Enhancement for Sparse-View 3D Gaussian Radiance Fields [73.49548565633123]
  Radiance fields represented by 3D Gaussians excel at synthesizing novel views, offering both high training efficiency and fast rendering. Existing methods often incorporate depth priors from dense estimation networks but overlook the inherent multi-view consistency in input images. We propose a view framework based on 3D Gaussian Splatting, named MCGS, enabling scene reconstruction from sparse input views.
  arXiv  Detail & Related papers  (2024-10-15T08:39:05Z)
• Dense Point Clouds Matter: Dust-GS for Scene Reconstruction from Sparse Viewpoints [9.069919085326]
  3D Gaussian Splatting (3DGS) has demonstrated remarkable performance in scene synthesis and novel view synthesis tasks. In this study, we present Dust-GS, a novel framework specifically designed to overcome the limitations of 3DGS in sparse viewpoint conditions.
  arXiv  Detail & Related papers  (2024-09-13T07:59:15Z)
• LM-Gaussian: Boost Sparse-view 3D Gaussian Splatting with Large Model Priors [34.91966359570867]
  sparse-view reconstruction is inherently ill-posed and under-constrained. We introduce LM-Gaussian, a method capable of generating high-quality reconstructions from a limited number of images. Our approach significantly reduces the data acquisition requirements compared to previous 3DGS methods.
  arXiv  Detail & Related papers  (2024-09-05T12:09:02Z)
• FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting [58.41056963451056]
  We propose a few-shot view synthesis framework based on 3D Gaussian Splatting. This framework enables real-time and photo-realistic view synthesis with as few as three training views. FSGS achieves state-of-the-art performance in both accuracy and rendering efficiency across diverse datasets.
  arXiv  Detail & Related papers  (2023-12-01T09:30:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.