SCALAR-NeRF: SCAlable LARge-scale Neural Radiance Fields for Scene Reconstruction

• URL: http://arxiv.org/abs/2311.16657v1
• Date: Tue, 28 Nov 2023 10:18:16 GMT
• Title: SCALAR-NeRF: SCAlable LARge-scale Neural Radiance Fields for Scene Reconstruction
• Authors: Yu Chen, Gim Hee Lee
• Abstract summary: We introduce SCALAR-NeRF, a novel framework tailored for scalable large-scale neural scene reconstruction. We structure the neural representation as an encoder-decoder architecture, where the encoder processes 3D point coordinates to produce encoded features. We propose an effective and efficient methodology to fuse the outputs from these local models to attain the final reconstruction.
• Score: 66.69049158826677
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we introduce SCALAR-NeRF, a novel framework tailored for scalable large-scale neural scene reconstruction. We structure the neural representation as an encoder-decoder architecture, where the encoder processes 3D point coordinates to produce encoded features, and the decoder generates geometric values that include volume densities of signed distances and colors. Our approach first trains a coarse global model on the entire image dataset. Subsequently, we partition the images into smaller blocks using KMeans with each block being modeled by a dedicated local model. We enhance the overlapping regions across different blocks by scaling up the bounding boxes of each local block. Notably, the decoder from the global model is shared across distinct blocks and therefore promoting alignment in the feature space of local encoders. We propose an effective and efficient methodology to fuse the outputs from these local models to attain the final reconstruction. Employing this refined coarse-to-fine strategy, our method outperforms state-of-the-art NeRF methods and demonstrates scalability for large-scale scene reconstruction. The code will be available on our project page at https://aibluefisher.github.io/SCALAR-NeRF/

Related papers

• Optimizing 3D Geometry Reconstruction from Implicit Neural Representations [2.3940819037450987]
  Implicit neural representations have emerged as a powerful tool in learning 3D geometry. We present a novel approach that both reduces computational expenses and enhances the capture of fine details.
  arXiv  Detail & Related papers  (2024-10-16T16:36:23Z)
• Global-guided Focal Neural Radiance Field for Large-scale Scene Rendering [12.272724419136575]
  We present a global-guided focal neural radiance field (GF-NeRF) that achieves high-fidelity rendering of large-scale scenes. Our method achieves high-fidelity, natural rendering results on various types of large-scale datasets.
  arXiv  Detail & Related papers  (2024-03-19T15:45:54Z)
• Leveraging Neural Radiance Field in Descriptor Synthesis for Keypoints Scene Coordinate Regression [1.2974519529978974]
  This paper introduces a pipeline for keypoint descriptor synthesis using Neural Radiance Field (NeRF) generating novel poses and feeding them into a trained NeRF model to create new views, our approach enhances the KSCR's capabilities in data-scarce environments. The proposed system could significantly improve localization accuracy by up to 50% and cost only a fraction of time for data synthesis.
  arXiv  Detail & Related papers  (2024-03-15T13:40:37Z)
• Distance Weighted Trans Network for Image Completion [52.318730994423106]
  We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. CNNs are used to augment the local texture information of coarse priors. DWT blocks are used to recover certain coarse textures and coherent visual structures.
  arXiv  Detail & Related papers  (2023-10-11T12:46:11Z)
• Grid-guided Neural Radiance Fields for Large Urban Scenes [146.06368329445857]
  Recent approaches propose to geographically divide the scene and adopt multiple sub-NeRFs to model each region individually. An alternative solution is to use a feature grid representation, which is computationally efficient and can naturally scale to a large scene. We present a new framework that realizes high-fidelity rendering on large urban scenes while being computationally efficient.
  arXiv  Detail & Related papers  (2023-03-24T13:56:45Z)
• MISSU: 3D Medical Image Segmentation via Self-distilling TransUNet [55.16833099336073]
  We propose to self-distill a Transformer-based UNet for medical image segmentation. It simultaneously learns global semantic information and local spatial-detailed features. Our MISSU achieves the best performance over previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-06-02T07:38:53Z)
• Compressible-composable NeRF via Rank-residual Decomposition [21.92736190195887]
  Neural Radiance Field (NeRF) has emerged as a compelling method to represent 3D objects and scenes for photo-realistic rendering. We present a neural representation that enables efficient and convenient manipulation of models. Our method is able to achieve comparable rendering quality to state-of-the-art methods, while enabling extra capability of compression and composition.
  arXiv  Detail & Related papers  (2022-05-30T06:18:59Z)
• Learning Local Displacements for Point Cloud Completion [93.54286830844134]
  We propose a novel approach aimed at object and semantic scene completion from a partial scan represented as a 3D point cloud. Our architecture relies on three novel layers that are used successively within an encoder-decoder structure. We evaluate both architectures on object and indoor scene completion tasks, achieving state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-03-30T18:31:37Z)
• Small Lesion Segmentation in Brain MRIs with Subpixel Embedding [105.1223735549524]
  We present a method to segment MRI scans of the human brain into ischemic stroke lesion and normal tissues. We propose a neural network architecture in the form of a standard encoder-decoder where predictions are guided by a spatial expansion embedding network.
  arXiv  Detail & Related papers  (2021-09-18T00:21:17Z)
• Multi-scale Attention U-Net (MsAUNet): A Modified U-Net Architecture for Scene Segmentation [1.713291434132985]
  We propose a novel multi-scale attention network for scene segmentation by using contextual information from an image. This network can map local features with their global counterparts with improved accuracy and emphasize on discriminative image regions. We have evaluated our model on two standard datasets named PascalVOC2012 and ADE20k.
  arXiv  Detail & Related papers  (2020-09-15T08:03:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.