Radiant: Large-scale 3D Gaussian Rendering based on Hierarchical Framework

• URL: http://arxiv.org/abs/2412.05546v1
• Date: Sat, 07 Dec 2024 05:48:00 GMT
• Title: Radiant: Large-scale 3D Gaussian Rendering based on Hierarchical Framework
• Authors: Haosong Peng, Tianyu Qi, Yufeng Zhan, Hao Li, Yalun Dai, Yuanqing Xia,
• Abstract summary: We propose Radiant, a hierarchical 3DGS algorithm designed for large-scale scene reconstruction. We show that Radiant improved reconstruction quality by up to 25.7% and reduced up to 79.6% end-to-end latency.
• Score: 13.583584930991847
• License:
• Abstract: With the advancement of computer vision, the recently emerged 3D Gaussian Splatting (3DGS) has increasingly become a popular scene reconstruction algorithm due to its outstanding performance. Distributed 3DGS can efficiently utilize edge devices to directly train on the collected images, thereby offloading computational demands and enhancing efficiency. However, traditional distributed frameworks often overlook computational and communication challenges in real-world environments, hindering large-scale deployment and potentially posing privacy risks. In this paper, we propose Radiant, a hierarchical 3DGS algorithm designed for large-scale scene reconstruction that considers system heterogeneity, enhancing the model performance and training efficiency. Via extensive empirical study, we find that it is crucial to partition the regions for each edge appropriately and allocate varying camera positions to each device for image collection and training. The core of Radiant is partitioning regions based on heterogeneous environment information and allocating workloads to each device accordingly. Furthermore, we provide a 3DGS model aggregation algorithm that enhances the quality and ensures the continuity of models' boundaries. Finally, we develop a testbed, and experiments demonstrate that Radiant improved reconstruction quality by up to 25.7\% and reduced up to 79.6\% end-to-end latency.

Related papers

• Tortho-Gaussian: Splatting True Digital Orthophoto Maps [12.796166971391774]
  True Digital Orthophoto Maps (TDOMs) are essential products for digital twins and Geographic Information Systems (GIS) Traditionally, TDOM generation involves a complex set of traditional photogrammetric process, which may deteriorate due to various challenges. We introduce TOrtho-Gaussian, a novel method inspired by 3D Gaussian Splatting (3DGS) that generates TDOMs through splatting of optimized anisotropic Gaussian kernel.
  arXiv  Detail & Related papers  (2024-11-29T10:22:38Z)
• DGTR: Distributed Gaussian Turbo-Reconstruction for Sparse-View Vast Scenes [81.56206845824572]
  Novel-view synthesis (NVS) approaches play a critical role in vast scene reconstruction. Few-shot methods often struggle with poor reconstruction quality in vast environments. This paper presents DGTR, a novel distributed framework for efficient Gaussian reconstruction for sparse-view vast scenes.
  arXiv  Detail & Related papers  (2024-11-19T07:51:44Z)
• GUS-IR: Gaussian Splatting with Unified Shading for Inverse Rendering [83.69136534797686]
  We present GUS-IR, a novel framework designed to address the inverse rendering problem for complicated scenes featuring rough and glossy surfaces. This paper starts by analyzing and comparing two prominent shading techniques popularly used for inverse rendering, forward shading and deferred shading. We propose a unified shading solution that combines the advantages of both techniques for better decomposition.
  arXiv  Detail & Related papers  (2024-11-12T01:51:05Z)
• GaRField++: Reinforced Gaussian Radiance Fields for Large-Scale 3D Scene Reconstruction [1.7624442706463355]
  This paper proposes a novel framework for large-scale scene reconstruction based on 3D Gaussian splatting (3DGS) For tackling the scalability issue, we split the large scene into multiple cells, and the candidate point-cloud and camera views of each cell are correlated. We show that our method consistently generates more high-fidelity rendering results than state-of-the-art methods of large-scale scene reconstruction.
  arXiv  Detail & Related papers  (2024-09-19T13:43:31Z)
• LM-Gaussian: Boost Sparse-view 3D Gaussian Splatting with Large Model Priors [34.91966359570867]
  sparse-view reconstruction is inherently ill-posed and under-constrained. We introduce LM-Gaussian, a method capable of generating high-quality reconstructions from a limited number of images. Our approach significantly reduces the data acquisition requirements compared to previous 3DGS methods.
  arXiv  Detail & Related papers  (2024-09-05T12:09:02Z)
• Compact 3D Gaussian Splatting for Static and Dynamic Radiance Fields [13.729716867839509]
  We propose a learnable mask strategy that significantly reduces the number of Gaussians while preserving high performance. In addition, we propose a compact but effective representation of view-dependent color by employing a grid-based neural field. Our work provides a comprehensive framework for 3D scene representation, achieving high performance, fast training, compactness, and real-time rendering.
  arXiv  Detail & Related papers  (2024-08-07T14:56:34Z)
• EfficientGS: Streamlining Gaussian Splatting for Large-Scale High-Resolution Scene Representation [29.334665494061113]
  'EfficientGS' is an advanced approach that optimize 3DGS for high-resolution, large-scale scenes. We analyze the densification process in 3DGS and identify areas of Gaussian over-proliferation. We propose a selective strategy, limiting Gaussian increase to key redundant primitives, thereby enhancing the representational efficiency.
  arXiv  Detail & Related papers  (2024-04-19T10:32:30Z)
• GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation [85.15374487533643]
  We introduce GRM, a large-scale reconstructor capable of recovering a 3D asset from sparse-view images in around 0.1s. GRM is a feed-forward transformer-based model that efficiently incorporates multi-view information. We also showcase the potential of GRM in generative tasks, i.e., text-to-3D and image-to-3D, by integrating it with existing multi-view diffusion models.
  arXiv  Detail & Related papers  (2024-03-21T17:59:34Z)
• GGRt: Towards Pose-free Generalizable 3D Gaussian Splatting in Real-time [112.32349668385635]
  GGRt is a novel approach to generalizable novel view synthesis that alleviates the need for real camera poses. As the first pose-free generalizable 3D-GS framework, GGRt achieves inference at $ge$ 5 FPS and real-time rendering at $ge$ 100 FPS.
  arXiv  Detail & Related papers  (2024-03-15T09:47:35Z)
• Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering [71.44349029439944]
  Recent 3D Gaussian Splatting method has achieved the state-of-the-art rendering quality and speed. We introduce Scaffold-GS, which uses anchor points to distribute local 3D Gaussians. We show that our method effectively reduces redundant Gaussians while delivering high-quality rendering.
  arXiv  Detail & Related papers  (2023-11-30T17:58:57Z)
• Spatial-Spectral Residual Network for Hyperspectral Image Super-Resolution [82.1739023587565]
  We propose a novel spectral-spatial residual network for hyperspectral image super-resolution (SSRNet) Our method can effectively explore spatial-spectral information by using 3D convolution instead of 2D convolution, which enables the network to better extract potential information. In each unit, we employ spatial and temporal separable 3D convolution to extract spatial and spectral information, which not only reduces unaffordable memory usage and high computational cost, but also makes the network easier to train.
  arXiv  Detail & Related papers  (2020-01-14T03:34:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.