RetinaGS: Scalable Training for Dense Scene Rendering with Billion-Scale 3D Gaussians

• URL: http://arxiv.org/abs/2406.11836v2
• Date: Sat, 22 Jun 2024 20:06:25 GMT
• Title: RetinaGS: Scalable Training for Dense Scene Rendering with Billion-Scale 3D Gaussians
• Authors: Bingling Li, Shengyi Chen, Luchao Wang, Kaimin Liao, Sijie Yan, Yuanjun Xiong,
• Abstract summary: We design a general model parallel training method for 3DGS, named RetinaGS, which uses a proper rendering equation. We observe a clear positive trend of increasing visual quality when increasing primitive numbers with our method. We also demonstrate the first attempt at training a 3DGS model with more than one billion primitives on the full MatrixCity dataset.
• Score: 12.461531097629857
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In this work, we explore the possibility of training high-parameter 3D Gaussian splatting (3DGS) models on large-scale, high-resolution datasets. We design a general model parallel training method for 3DGS, named RetinaGS, which uses a proper rendering equation and can be applied to any scene and arbitrary distribution of Gaussian primitives. It enables us to explore the scaling behavior of 3DGS in terms of primitive numbers and training resolutions that were difficult to explore before and surpass previous state-of-the-art reconstruction quality. We observe a clear positive trend of increasing visual quality when increasing primitive numbers with our method. We also demonstrate the first attempt at training a 3DGS model with more than one billion primitives on the full MatrixCity dataset that attains a promising visual quality.

Related papers

• Self-Evolving Depth-Supervised 3D Gaussian Splatting from Rendered Stereo Pairs [27.364205809607302]
  3D Gaussian Splatting (GS) significantly struggles to accurately represent the underlying 3D scene geometry. We address this limitation, undertaking a comprehensive analysis of the integration of depth priors throughout the optimization process. This latter dynamically exploits depth cues from a readily available stereo network, processing virtual stereo pairs rendered by the GS model itself during training and achieving consistent self-improvement.
  arXiv  Detail & Related papers  (2024-09-11T17:59:58Z)
• ShapeSplat: A Large-scale Dataset of Gaussian Splats and Their Self-Supervised Pretraining [104.34751911174196]
  We build a large-scale dataset of 3DGS using ShapeNet and ModelNet datasets. Our dataset ShapeSplat consists of 65K objects from 87 unique categories. We introduce textbftextitGaussian-MAE, which highlights the unique benefits of representation learning from Gaussian parameters.
  arXiv  Detail & Related papers  (2024-08-20T14:49:14Z)
• GaussianStego: A Generalizable Stenography Pipeline for Generative 3D Gaussians Splatting [38.33958617286536]
  GaussianStego is a method for embedding steganographic information in the rendering of generated 3D assets. Our approach employs an optimization framework that enables the accurate extraction of hidden information.
  arXiv  Detail & Related papers  (2024-07-01T13:57:44Z)
• Effective Rank Analysis and Regularization for Enhanced 3D Gaussian Splatting [33.01987451251659]
  3D Gaussian Splatting (3DGS) has emerged as a promising technique capable of real-time rendering with high-quality 3D reconstruction. Despite its potential, 3DGS encounters challenges, including needle-like artifacts, suboptimal geometries, and inaccurate normals. We introduce effective rank as a regularization, which constrains the structure of the Gaussians.
  arXiv  Detail & Related papers  (2024-06-17T15:51:59Z)
• PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting [59.277480452459315]
  We propose a principled spatial sensitivity pruning score that outperforms current approaches. We also propose a multi-round prune-refine pipeline that can be applied to any pretrained 3D-GS model. Our pipeline increases the average rendering speed of 3D-GS by 2.65$times$ while retaining more salient foreground information.
  arXiv  Detail & Related papers  (2024-06-14T17:53:55Z)
• LP-3DGS: Learning to Prune 3D Gaussian Splatting [71.97762528812187]
  We propose learning-to-prune 3DGS, where a trainable binary mask is applied to the importance score that can find optimal pruning ratio automatically. Experiments have shown that LP-3DGS consistently produces a good balance that is both efficient and high quality.
  arXiv  Detail & Related papers  (2024-05-29T05:58:34Z)
• DOGS: Distributed-Oriented Gaussian Splatting for Large-Scale 3D Reconstruction Via Gaussian Consensus [56.45194233357833]
  We propose DoGaussian, a method that trains 3DGS distributedly. Our method accelerates the training of 3DGS by 6+ times when evaluated on large-scale scenes.
  arXiv  Detail & Related papers  (2024-05-22T19:17:58Z)
• latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstruction [48.86083272054711]
  latentSplat is a method to predict semantic Gaussians in a 3D latent space that can be splatted and decoded by a light-weight generative 2D architecture. We show that latentSplat outperforms previous works in reconstruction quality and generalization, while being fast and scalable to high-resolution data.
  arXiv  Detail & Related papers  (2024-03-24T20:48:36Z)
• GaussianPro: 3D Gaussian Splatting with Progressive Propagation [49.918797726059545]
  3DGS relies heavily on the point cloud produced by Structure-from-Motion (SfM) techniques. We propose a novel method that applies a progressive propagation strategy to guide the densification of the 3D Gaussians. Our method significantly surpasses 3DGS on the dataset, exhibiting an improvement of 1.15dB in terms of PSNR.
  arXiv  Detail & Related papers  (2024-02-22T16:00:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.