Efficient Large-scale Scene Representation with a Hybrid of High-resolution Grid and Plane Features

• URL: http://arxiv.org/abs/2303.03003v2
• Date: Tue, 7 Mar 2023 14:46:21 GMT
• Title: Efficient Large-scale Scene Representation with a Hybrid of High-resolution Grid and Plane Features
• Authors: Yuqi Zhang, Guanying Chen and Shuguang Cui
• Abstract summary: Existing neural radiance fields (NeRF) methods for large-scale scene modeling require days of training using multiple GPUs. We introduce a new and efficient hybrid feature representation for NeRF that fuses the 3D hash-grids and high-resolution 2D dense plane features. Based on this hybrid representation, we propose a fast optimization NeRF variant, called GP-NeRF, that achieves better rendering results while maintaining a compact model size.
• Score: 44.25307397334988
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing neural radiance fields (NeRF) methods for large-scale scene modeling require days of training using multiple GPUs, hindering their applications in scenarios with limited computing resources. Despite fast optimization NeRF variants have been proposed based on the explicit dense or hash grid features, their effectivenesses are mainly demonstrated in object-scale scene representation. In this paper, we point out that the low feature resolution in explicit representation is the bottleneck for large-scale unbounded scene representation. To address this problem, we introduce a new and efficient hybrid feature representation for NeRF that fuses the 3D hash-grids and high-resolution 2D dense plane features. Compared with the dense-grid representation, the resolution of a dense 2D plane can be scaled up more efficiently. Based on this hybrid representation, we propose a fast optimization NeRF variant, called GP-NeRF, that achieves better rendering results while maintaining a compact model size. Extensive experiments on multiple large-scale unbounded scene datasets show that our model can converge in 1.5 hours using a single GPU while achieving results comparable to or even better than the existing method that requires about one day's training with 8 GPUs.

Related papers

• Aerial-NeRF: Adaptive Spatial Partitioning and Sampling for Large-Scale Aerial Rendering [10.340739248752516]
  We propose Aerial-NeRF to render complex aerial scenes with high-precision. Our model allows us to perform rendering over 4 times as fast as compared to multiple competitors. New state-of-the-art results have been achieved on two public large-scale aerial datasets.
  arXiv  Detail & Related papers  (2024-05-10T02:57:02Z)
• VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction [59.40711222096875]
  We present VastGaussian, the first method for high-quality reconstruction and real-time rendering on large scenes based on 3D Gaussian Splatting. Our approach outperforms existing NeRF-based methods and achieves state-of-the-art results on multiple large scene datasets.
  arXiv  Detail & Related papers  (2024-02-27T11:40:50Z)
• HybridNeRF: Efficient Neural Rendering via Adaptive Volumetric Surfaces [71.1071688018433]
  Neural radiance fields provide state-of-the-art view synthesis quality but tend to be slow to render. We propose a method, HybridNeRF, that leverages the strengths of both representations by rendering most objects as surfaces. We improve error rates by 15-30% while achieving real-time framerates (at least 36 FPS) for virtual-reality resolutions (2Kx2K)
  arXiv  Detail & Related papers  (2023-12-05T22:04:49Z)
• Adaptive Multi-NeRF: Exploit Efficient Parallelism in Adaptive Multiple Scale Neural Radiance Field Rendering [3.8200916793910973]
  Recent advances in Neural Radiance Fields (NeRF) have demonstrated significant potential for representing 3D scene appearances as implicit neural networks. However, the lengthy training and rendering process hinders the widespread adoption of this promising technique for real-time rendering applications. We present an effective adaptive multi-NeRF method designed to accelerate the neural rendering process for large scenes.
  arXiv  Detail & Related papers  (2023-10-03T08:34:49Z)
• Grid-guided Neural Radiance Fields for Large Urban Scenes [146.06368329445857]
  Recent approaches propose to geographically divide the scene and adopt multiple sub-NeRFs to model each region individually. An alternative solution is to use a feature grid representation, which is computationally efficient and can naturally scale to a large scene. We present a new framework that realizes high-fidelity rendering on large urban scenes while being computationally efficient.
  arXiv  Detail & Related papers  (2023-03-24T13:56:45Z)
• Magic3D: High-Resolution Text-to-3D Content Creation [78.40092800817311]
  DreamFusion has recently demonstrated the utility of a pre-trained text-to-image diffusion model to optimize Neural Radiance Fields (NeRF) In this paper, we address these limitations by utilizing a two-stage optimization framework. Our method, dubbed Magic3D, can create high quality 3D mesh models in 40 minutes, which is 2x faster than DreamFusion.
  arXiv  Detail & Related papers  (2022-11-18T18:59:59Z)
• Neural Deformable Voxel Grid for Fast Optimization of Dynamic View Synthesis [63.25919018001152]
  We propose a fast deformable radiance field method to handle dynamic scenes. Our method achieves comparable performance to D-NeRF using only 20 minutes for training.
  arXiv  Detail & Related papers  (2022-06-15T17:49:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.