Neural Adaptive SCEne Tracing

• URL: http://arxiv.org/abs/2202.13664v1
• Date: Mon, 28 Feb 2022 10:27:23 GMT
• Title: Neural Adaptive SCEne Tracing
• Authors: Rui Li, Darius R\"Uckert, Yuanhao Wang, Ramzi Idoughi, Wolfgang Heidrich
• Abstract summary: We present NAScenT, the first neural rendering method based on directly training a hybrid explicit-implicit neural representation. NAScenT is capable of reconstructing challenging scenes including both large, sparsely populated volumes like UAV captured outdoor environments.
• Score: 24.781844909539686
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural rendering with implicit neural networks has recently emerged as an attractive proposition for scene reconstruction, achieving excellent quality albeit at high computational cost. While the most recent generation of such methods has made progress on the rendering (inference) times, very little progress has been made on improving the reconstruction (training) times. In this work, we present Neural Adaptive Scene Tracing (NAScenT), the first neural rendering method based on directly training a hybrid explicit-implicit neural representation. NAScenT uses a hierarchical octree representation with one neural network per leaf node and combines this representation with a two-stage sampling process that concentrates ray samples where they matter most near object surfaces. As a result, NAScenT is capable of reconstructing challenging scenes including both large, sparsely populated volumes like UAV captured outdoor environments, as well as small scenes with high geometric complexity. NAScenT outperforms existing neural rendering approaches in terms of both quality and training time.

Related papers

• An experimental comparative study of backpropagation and alternatives for training binary neural networks for image classification [1.0749601922718608]
  Binary neural networks promise to reduce the size of deep neural network models. They may allow the deployment of more powerful models on edge devices. However, binary neural networks are still proven to be difficult to train using the backpropagation-based gradient descent scheme.
  arXiv  Detail & Related papers  (2024-08-08T13:39:09Z)
• D-NPC: Dynamic Neural Point Clouds for Non-Rigid View Synthesis from Monocular Video [53.83936023443193]
  This paper contributes to the field by introducing a new synthesis method for dynamic novel view from monocular video, such as smartphone captures. Our approach represents the as a $textitdynamic neural point cloud$, an implicit time-conditioned point cloud that encodes local geometry and appearance in separate hash-encoded neural feature grids.
  arXiv  Detail & Related papers  (2024-06-14T14:35:44Z)
• DNS SLAM: Dense Neural Semantic-Informed SLAM [92.39687553022605]
  DNS SLAM is a novel neural RGB-D semantic SLAM approach featuring a hybrid representation. Our method integrates multi-view geometry constraints with image-based feature extraction to improve appearance details. Our experimental results achieve state-of-the-art performance on both synthetic data and real-world data tracking.
  arXiv  Detail & Related papers  (2023-11-30T21:34:44Z)
• Volume Feature Rendering for Fast Neural Radiance Field Reconstruction [11.05302598034426]
  Neural radiance fields (NeRFs) are able to synthesize realistic novel views from multi-view images captured from distinct positions and perspectives. In NeRF's rendering pipeline, neural networks are used to represent a scene independently or transform queried learnable feature vector of a point to the expected color or density. We propose to render the queried feature vectors of a ray first and then transform the rendered feature vector to the final pixel color by a neural network.
  arXiv  Detail & Related papers  (2023-05-29T06:58:27Z)
• NeRN -- Learning Neural Representations for Neural Networks [3.7384109981836153]
  We show that, when adapted correctly, neural representations can be used to represent the weights of a pre-trained convolutional neural network. Inspired by coordinate inputs of previous neural representation methods, we assign a coordinate to each convolutional kernel in our network. We present two applications using NeRN, demonstrating the capabilities of the learned representations.
  arXiv  Detail & Related papers  (2022-12-27T17:14:44Z)
• AdaNeRF: Adaptive Sampling for Real-time Rendering of Neural Radiance Fields [8.214695794896127]
  Novel view synthesis has recently been revolutionized by learning neural radiance fields directly from sparse observations. rendering images with this new paradigm is slow due to the fact that an accurate quadrature of the volume rendering equation requires a large number of samples for each ray. We propose a novel dual-network architecture that takes an direction by learning how to best reduce the number of required sample points.
  arXiv  Detail & Related papers  (2022-07-21T05:59:13Z)
• Neural Rays for Occlusion-aware Image-based Rendering [108.34004858785896]
  We present a new neural representation, called Neural Ray (NeuRay), for the novel view synthesis (NVS) task with multi-view images as input. NeuRay can quickly generate high-quality novel view rendering images of unseen scenes with little finetuning.
  arXiv  Detail & Related papers  (2021-07-28T15:09:40Z)
• Fast Training of Neural Lumigraph Representations using Meta Learning [109.92233234681319]
  We develop a new neural rendering approach with the goal of quickly learning a high-quality representation which can also be rendered in real-time. Our approach, MetaNLR++, accomplishes this by using a unique combination of a neural shape representation and 2D CNN-based image feature extraction, aggregation, and re-projection. We show that MetaNLR++ achieves similar or better photorealistic novel view synthesis results in a fraction of the time that competing methods require.
  arXiv  Detail & Related papers  (2021-06-28T18:55:50Z)
• NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction [88.02850205432763]
  We present a novel neural surface reconstruction method, called NeuS, for reconstructing objects and scenes with high fidelity from 2D image inputs. Existing neural surface reconstruction approaches, such as DVR and IDR, require foreground mask as supervision. We observe that the conventional volume rendering method causes inherent geometric errors for surface reconstruction. We propose a new formulation that is free of bias in the first order of approximation, thus leading to more accurate surface reconstruction even without the mask supervision.
  arXiv  Detail & Related papers  (2021-06-20T12:59:42Z)
• MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo [52.329580781898116]
  We present MVSNeRF, a novel neural rendering approach that can efficiently reconstruct neural radiance fields for view synthesis. Unlike prior works on neural radiance fields that consider per-scene optimization on densely captured images, we propose a generic deep neural network that can reconstruct radiance fields from only three nearby input views via fast network inference.
  arXiv  Detail & Related papers  (2021-03-29T13:15:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.