UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction

• URL: http://arxiv.org/abs/2104.10078v1
• Date: Tue, 20 Apr 2021 15:59:38 GMT
• Title: UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction
• Authors: Michael Oechsle, Songyou Peng, Andreas Geiger
• Abstract summary: We present a novel method for reconstructing surfaces from multi-view images using Neural implicit 3D representations. Our key insight is that implicit surface models and radiance fields can be formulated in a unified way, enabling both surface and volume rendering. Our experiments demonstrate that we outperform NeRF in terms of reconstruction quality while performing on par with IDR without requiring masks.
• Score: 61.17219252031391
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Neural implicit 3D representations have emerged as a powerful paradigm for reconstructing surfaces from multi-view images and synthesizing novel views. Unfortunately, existing methods such as DVR or IDR require accurate per-pixel object masks as supervision. At the same time, neural radiance fields have revolutionized novel view synthesis. However, NeRF's estimated volume density does not admit accurate surface reconstruction. Our key insight is that implicit surface models and radiance fields can be formulated in a unified way, enabling both surface and volume rendering using the same model. This unified perspective enables novel, more efficient sampling procedures and the ability to reconstruct accurate surfaces without input masks. We compare our method on the DTU, BlendedMVS, and a synthetic indoor dataset. Our experiments demonstrate that we outperform NeRF in terms of reconstruction quality while performing on par with IDR without requiring masks.

Related papers

• High-Fidelity Mask-free Neural Surface Reconstruction for Virtual Reality [6.987660269386849]
  Hi-NeuS is a novel rendering-based framework for neural implicit surface reconstruction. Our approach has been validated through NeuS and its variant Neuralangelo.
  arXiv  Detail & Related papers  (2024-09-20T02:07:49Z)
• HybridNeRF: Efficient Neural Rendering via Adaptive Volumetric Surfaces [71.1071688018433]
  Neural radiance fields provide state-of-the-art view synthesis quality but tend to be slow to render. We propose a method, HybridNeRF, that leverages the strengths of both representations by rendering most objects as surfaces. We improve error rates by 15-30% while achieving real-time framerates (at least 36 FPS) for virtual-reality resolutions (2Kx2K)
  arXiv  Detail & Related papers  (2023-12-05T22:04:49Z)
• Multi-Space Neural Radiance Fields [74.46513422075438]
  Existing Neural Radiance Fields (NeRF) methods suffer from the existence of reflective objects. We propose a multi-space neural radiance field (MS-NeRF) that represents the scene using a group of feature fields in parallel sub-spaces. Our approach significantly outperforms the existing single-space NeRF methods for rendering high-quality scenes.
  arXiv  Detail & Related papers  (2023-05-07T13:11:07Z)
• Learning Neural Duplex Radiance Fields for Real-Time View Synthesis [33.54507228895688]
  We propose a novel approach to distill and bake NeRFs into highly efficient mesh-based neural representations. We demonstrate the effectiveness and superiority of our approach via extensive experiments on a range of standard datasets.
  arXiv  Detail & Related papers  (2023-04-20T17:59:52Z)
• NeAT: Learning Neural Implicit Surfaces with Arbitrary Topologies from Multi-view Images [17.637064969966847]
  NeAT is a new neural rendering framework that learns implicit surfaces with arbitrary topologies from multi-view images. NeAT supports easy field-to-mesh conversion using the classic Marching Cubes algorithm. Our approach is able to faithfully reconstruct both watertight and non-watertight surfaces.
  arXiv  Detail & Related papers  (2023-03-21T16:49:41Z)
• NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild [80.09093712055682]
  We introduce a surface analog of implicit models called Neural Reflectance Surfaces (NeRS) NeRS learns a neural shape representation of a closed surface that is diffeomorphic to a sphere, guaranteeing water-tight reconstructions. We demonstrate that surface-based neural reconstructions enable learning from such data, outperforming volumetric neural rendering-based reconstructions.
  arXiv  Detail & Related papers  (2021-10-14T17:59:58Z)
• NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction [88.02850205432763]
  We present a novel neural surface reconstruction method, called NeuS, for reconstructing objects and scenes with high fidelity from 2D image inputs. Existing neural surface reconstruction approaches, such as DVR and IDR, require foreground mask as supervision. We observe that the conventional volume rendering method causes inherent geometric errors for surface reconstruction. We propose a new formulation that is free of bias in the first order of approximation, thus leading to more accurate surface reconstruction even without the mask supervision.
  arXiv  Detail & Related papers  (2021-06-20T12:59:42Z)
• MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo [52.329580781898116]
  We present MVSNeRF, a novel neural rendering approach that can efficiently reconstruct neural radiance fields for view synthesis. Unlike prior works on neural radiance fields that consider per-scene optimization on densely captured images, we propose a generic deep neural network that can reconstruct radiance fields from only three nearby input views via fast network inference.
  arXiv  Detail & Related papers  (2021-03-29T13:15:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.