Global-guided Focal Neural Radiance Field for Large-scale Scene Rendering

• URL: http://arxiv.org/abs/2403.12839v2
• Date: Fri, 13 Sep 2024 05:44:20 GMT
• Title: Global-guided Focal Neural Radiance Field for Large-scale Scene Rendering
• Authors: Mingqi Shao, Feng Xiong, Hang Zhang, Shuang Yang, Mu Xu, Wei Bian, Xueqian Wang,
• Abstract summary: We present a global-guided focal neural radiance field (GF-NeRF) that achieves high-fidelity rendering of large-scale scenes. Our method achieves high-fidelity, natural rendering results on various types of large-scale datasets.
• Score: 12.272724419136575
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural radiance fields~(NeRF) have recently been applied to render large-scale scenes. However, their limited model capacity typically results in blurred rendering results. Existing large-scale NeRFs primarily address this limitation by partitioning the scene into blocks, which are subsequently handled by separate sub-NeRFs. These sub-NeRFs, trained from scratch and processed independently, lead to inconsistencies in geometry and appearance across the scene. Consequently, the rendering quality fails to exhibit significant improvement despite the expansion of model capacity. In this work, we present global-guided focal neural radiance field (GF-NeRF) that achieves high-fidelity rendering of large-scale scenes. Our proposed GF-NeRF utilizes a two-stage (Global and Focal) architecture and a global-guided training strategy. The global stage obtains a continuous representation of the entire scene while the focal stage decomposes the scene into multiple blocks and further processes them with distinct sub-encoders. Leveraging this two-stage architecture, sub-encoders only need fine-tuning based on the global encoder, thus reducing training complexity in the focal stage while maintaining scene-wide consistency. Spatial information and error information from the global stage also benefit the sub-encoders to focus on crucial areas and effectively capture more details of large-scale scenes. Notably, our approach does not rely on any prior knowledge about the target scene, attributing GF-NeRF adaptable to various large-scale scene types, including street-view and aerial-view scenes. We demonstrate that our method achieves high-fidelity, natural rendering results on various types of large-scale datasets. Our project page: https://shaomq2187.github.io/GF-NeRF/

Related papers

• InterNeRF: Scaling Radiance Fields via Parameter Interpolation [36.014610797521605]
  We propose InterNeRF, a novel architecture for rendering a target view using a subset of the model's parameters. We demonstrate significant improvements in multi-room scenes while remaining competitive on standard benchmarks.
  arXiv  Detail & Related papers  (2024-06-17T16:55:22Z)
• Aerial-NeRF: Adaptive Spatial Partitioning and Sampling for Large-Scale Aerial Rendering [10.340739248752516]
  We propose Aerial-NeRF to render complex aerial scenes with high-precision. Our model allows us to perform rendering over 4 times as fast as compared to multiple competitors. New state-of-the-art results have been achieved on two public large-scale aerial datasets.
  arXiv  Detail & Related papers  (2024-05-10T02:57:02Z)
• SCALAR-NeRF: SCAlable LARge-scale Neural Radiance Fields for Scene Reconstruction [66.69049158826677]
  We introduce SCALAR-NeRF, a novel framework tailored for scalable large-scale neural scene reconstruction. We structure the neural representation as an encoder-decoder architecture, where the encoder processes 3D point coordinates to produce encoded features. We propose an effective and efficient methodology to fuse the outputs from these local models to attain the final reconstruction.
  arXiv  Detail & Related papers  (2023-11-28T10:18:16Z)
• Grid-guided Neural Radiance Fields for Large Urban Scenes [146.06368329445857]
  Recent approaches propose to geographically divide the scene and adopt multiple sub-NeRFs to model each region individually. An alternative solution is to use a feature grid representation, which is computationally efficient and can naturally scale to a large scene. We present a new framework that realizes high-fidelity rendering on large urban scenes while being computationally efficient.
  arXiv  Detail & Related papers  (2023-03-24T13:56:45Z)
• CLONeR: Camera-Lidar Fusion for Occupancy Grid-aided Neural Representations [77.90883737693325]
  This paper proposes CLONeR, which significantly improves upon NeRF by allowing it to model large outdoor driving scenes observed from sparse input sensor views. This is achieved by decoupling occupancy and color learning within the NeRF framework into separate Multi-Layer Perceptrons (MLPs) trained using LiDAR and camera data, respectively. In addition, this paper proposes a novel method to build differentiable 3D Occupancy Grid Maps (OGM) alongside the NeRF model, and leverage this occupancy grid for improved sampling of points along a ray for rendering in metric space.
  arXiv  Detail & Related papers  (2022-09-02T17:44:50Z)
• NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction [50.54946139497575]
  We propose NeRFusion, a method that combines the advantages of NeRF and TSDF-based fusion techniques to achieve efficient large-scale reconstruction and photo-realistic rendering. We demonstrate that NeRFusion achieves state-of-the-art quality on both large-scale indoor and small-scale object scenes, with substantially faster reconstruction than NeRF and other recent methods.
  arXiv  Detail & Related papers  (2022-03-21T18:56:35Z)
• Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs [54.41204057689033]
  We explore how to leverage neural fields (NeRFs) to build interactive 3D environments from large-scale visual captures spanning buildings or even multiple city blocks collected primarily from drone data. In contrast to the single object scenes against which NeRFs have been traditionally evaluated, this setting poses multiple challenges. We introduce a simple clustering algorithm that partitions training images (or rather pixels) into different NeRF submodules that can be trained in parallel.
  arXiv  Detail & Related papers  (2021-12-20T17:40:48Z)
• BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering [145.95688637309746]
  We introduce BungeeNeRF, a progressive neural radiance field that achieves level-of-detail rendering across drastically varied scales. We demonstrate the superiority of BungeeNeRF in modeling diverse multi-scale scenes with drastically varying views on multiple data sources.
  arXiv  Detail & Related papers  (2021-12-10T13:16:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.