MCGS: Multiview Consistency Enhancement for Sparse-View 3D Gaussian Radiance Fields

• URL: http://arxiv.org/abs/2410.11394v1
• Date: Tue, 15 Oct 2024 08:39:05 GMT
• Title: MCGS: Multiview Consistency Enhancement for Sparse-View 3D Gaussian Radiance Fields
• Authors: Yuru Xiao, Deming Zhai, Wenbo Zhao, Kui Jiang, Junjun Jiang, Xianming Liu,
• Abstract summary: Radiance fields represented by 3D Gaussians excel at synthesizing novel views, offering both high training efficiency and fast rendering. Existing methods often incorporate depth priors from dense estimation networks but overlook the inherent multi-view consistency in input images. We propose a view framework based on 3D Gaussian Splatting, named MCGS, enabling scene reconstruction from sparse input views.
• Score: 73.49548565633123
• License:
• Abstract: Radiance fields represented by 3D Gaussians excel at synthesizing novel views, offering both high training efficiency and fast rendering. However, with sparse input views, the lack of multi-view consistency constraints results in poorly initialized point clouds and unreliable heuristics for optimization and densification, leading to suboptimal performance. Existing methods often incorporate depth priors from dense estimation networks but overlook the inherent multi-view consistency in input images. Additionally, they rely on multi-view stereo (MVS)-based initialization, which limits the efficiency of scene representation. To overcome these challenges, we propose a view synthesis framework based on 3D Gaussian Splatting, named MCGS, enabling photorealistic scene reconstruction from sparse input views. The key innovations of MCGS in enhancing multi-view consistency are as follows: i) We introduce an initialization method by leveraging a sparse matcher combined with a random filling strategy, yielding a compact yet sufficient set of initial points. This approach enhances the initial geometry prior, promoting efficient scene representation. ii) We develop a multi-view consistency-guided progressive pruning strategy to refine the Gaussian field by strengthening consistency and eliminating low-contribution Gaussians. These modular, plug-and-play strategies enhance robustness to sparse input views, accelerate rendering, and reduce memory consumption, making MCGS a practical and efficient framework for 3D Gaussian Splatting.

Related papers

• Mini-Splatting2: Building 360 Scenes within Minutes via Aggressive Gaussian Densification [4.733612131945549]
  Mini-Splatting2 achieves a balanced trade-off among optimization time, the number of Gaussians, and rendering quality. Our work sets the stage for more efficient, high-quality 3D scene modeling in real-world applications.
  arXiv  Detail & Related papers  (2024-11-19T11:47:40Z)
• DGTR: Distributed Gaussian Turbo-Reconstruction for Sparse-View Vast Scenes [81.56206845824572]
  Novel-view synthesis (NVS) approaches play a critical role in vast scene reconstruction. Few-shot methods often struggle with poor reconstruction quality in vast environments. This paper presents DGTR, a novel distributed framework for efficient Gaussian reconstruction for sparse-view vast scenes.
  arXiv  Detail & Related papers  (2024-11-19T07:51:44Z)
• MVGS: Multi-view-regulated Gaussian Splatting for Novel View Synthesis [22.80370814838661]
  Recent works in volume rendering, textite.g. NeRF and 3D Gaussian Splatting (3DGS), significantly advance the rendering quality and efficiency. We propose a new 3DGS optimization method embodying four key novel contributions.
  arXiv  Detail & Related papers  (2024-10-02T23:48:31Z)
• Self-augmented Gaussian Splatting with Structure-aware Masks for Sparse-view 3D Reconstruction [9.953394373473621]
  Sparse-view 3D reconstruction is a formidable challenge in computer vision. We propose a self-augmented coarse-to-fine Gaussian splatting paradigm, enhanced with a structure-aware mask. Our method achieves state-of-the-art performances for sparse input views in both perceptual quality and efficiency.
  arXiv  Detail & Related papers  (2024-08-09T03:09:22Z)
• FreeSplat: Generalizable 3D Gaussian Splatting Towards Free-View Synthesis of Indoor Scenes [50.534213038479926]
  FreeSplat is capable of reconstructing geometrically consistent 3D scenes from long sequence input towards free-view synthesis. We propose a simple but effective free-view training strategy that ensures robust view synthesis across broader view range regardless of the number of views.
  arXiv  Detail & Related papers  (2024-05-28T08:40:14Z)
• MVSGaussian: Fast Generalizable Gaussian Splatting Reconstruction from Multi-View Stereo [54.00987996368157]
  We present MVSGaussian, a new generalizable 3D Gaussian representation approach derived from Multi-View Stereo (MVS) MVSGaussian achieves real-time rendering with better synthesis quality for each scene.
  arXiv  Detail & Related papers  (2024-05-20T17:59:30Z)
• CoherentGS: Sparse Novel View Synthesis with Coherent 3D Gaussians [18.42203035154126]
  We introduce a structured Gaussian representation that can be controlled in 2D image space. We then constraint the Gaussians, in particular their position, and prevent them from moving independently during optimization. We demonstrate significant improvements compared to the state-of-the-art sparse-view NeRF-based approaches on a variety of scenes.
  arXiv  Detail & Related papers  (2024-03-28T15:27:13Z)
• FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting [58.41056963451056]
  We propose a few-shot view synthesis framework based on 3D Gaussian Splatting. This framework enables real-time and photo-realistic view synthesis with as few as three training views. FSGS achieves state-of-the-art performance in both accuracy and rendering efficiency across diverse datasets.
  arXiv  Detail & Related papers  (2023-12-01T09:30:02Z)
• Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering [71.44349029439944]
  Recent 3D Gaussian Splatting method has achieved the state-of-the-art rendering quality and speed. We introduce Scaffold-GS, which uses anchor points to distribute local 3D Gaussians. We show that our method effectively reduces redundant Gaussians while delivering high-quality rendering.
  arXiv  Detail & Related papers  (2023-11-30T17:58:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.