NeurMiPs: Neural Mixture of Planar Experts for View Synthesis

• URL: http://arxiv.org/abs/2204.13696v1
• Date: Thu, 28 Apr 2022 17:59:41 GMT
• Title: NeurMiPs: Neural Mixture of Planar Experts for View Synthesis
• Authors: Zhi-Hao Lin, Wei-Chiu Ma, Hao-Yu Hsu, Yu-Chiang Frank Wang, Shenlong Wang
• Abstract summary: NeurMiPs is a novel planar-based scene representation for modeling geometry and appearance. We render novel views by calculating ray-plane intersections and composite output colors and densities at intersected points to the image. Experiments demonstrate superior performance and speed of our proposed method, compared to other 3D representations in novel view synthesis.
• Score: 49.25559264261876
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Neural Mixtures of Planar Experts (NeurMiPs), a novel planar-based scene representation for modeling geometry and appearance. NeurMiPs leverages a collection of local planar experts in 3D space as the scene representation. Each planar expert consists of the parameters of the local rectangular shape representing geometry and a neural radiance field modeling the color and opacity. We render novel views by calculating ray-plane intersections and composite output colors and densities at intersected points to the image. NeurMiPs blends the efficiency of explicit mesh rendering and flexibility of the neural radiance field. Experiments demonstrate superior performance and speed of our proposed method, compared to other 3D representations in novel view synthesis.

Related papers

• Depth Reconstruction with Neural Signed Distance Fields in Structured Light Systems [15.603880588503355]
  We introduce a novel depth estimation technique for multi-frame structured light setups using neural implicit representations of 3D space. Our approach employs a neural signed distance field (SDF), trained through self-supervised differentiable rendering.
  arXiv  Detail & Related papers  (2024-05-20T13:24:35Z)
• Anti-Aliased Neural Implicit Surfaces with Encoding Level of Detail [54.03399077258403]
  We present LoD-NeuS, an efficient neural representation for high-frequency geometry detail recovery and anti-aliased novel view rendering. Our representation aggregates space features from a multi-convolved featurization within a conical frustum along a ray.
  arXiv  Detail & Related papers  (2023-09-19T05:44:00Z)
• NeRFMeshing: Distilling Neural Radiance Fields into Geometrically-Accurate 3D Meshes [56.31855837632735]
  We propose a compact and flexible architecture that enables easy 3D surface reconstruction from any NeRF-driven approach. Our final 3D mesh is physically accurate and can be rendered in real time on an array of devices.
  arXiv  Detail & Related papers  (2023-03-16T16:06:03Z)
• Multi-Plane Neural Radiance Fields for Novel View Synthesis [5.478764356647437]
  Novel view synthesis is a long-standing problem that revolves around rendering frames of scenes from novel camera viewpoints. In this work, we examine the performance, generalization, and efficiency of single-view multi-plane neural radiance fields. We propose a new multiplane NeRF architecture that accepts multiple views to improve the synthesis results and expand the viewing range.
  arXiv  Detail & Related papers  (2023-03-03T06:32:55Z)
• Coordinates Are NOT Lonely -- Codebook Prior Helps Implicit Neural 3D Representations [29.756718435405983]
  Implicit neural 3D representation has achieved impressive results in surface or scene reconstruction and novel view synthesis. Existing approaches, such as Neural Radiance Field (NeRF) and its variants, usually require dense input views. We introduce a novel coordinate-based model, CoCo-INR, for implicit neural 3D representation.
  arXiv  Detail & Related papers  (2022-10-20T11:13:50Z)
• Urban Radiance Fields [77.43604458481637]
  We perform 3D reconstruction and novel view synthesis from data captured by scanning platforms commonly deployed for world mapping in urban outdoor environments. Our approach extends Neural Radiance Fields, which has been demonstrated to synthesize realistic novel images for small scenes in controlled settings. Each of these three extensions provides significant performance improvements in experiments on Street View data.
  arXiv  Detail & Related papers  (2021-11-29T15:58:16Z)
• Extracting Triangular 3D Models, Materials, and Lighting From Images [59.33666140713829]
  We present an efficient method for joint optimization of materials and lighting from multi-view image observations. We leverage meshes with spatially-varying materials and environment that can be deployed in any traditional graphics engine.
  arXiv  Detail & Related papers  (2021-11-24T13:58:20Z)
• Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering [60.02806355570514]
  Inferring representations of 3D scenes from 2D observations is a fundamental problem of computer graphics, computer vision, and artificial intelligence. We propose a novel neural scene representation, Light Field Networks or LFNs, which represent both geometry and appearance of the underlying 3D scene in a 360-degree, four-dimensional light field. Rendering a ray from an LFN requires only a *single* network evaluation, as opposed to hundreds of evaluations per ray for ray-marching or based on volumetrics.
  arXiv  Detail & Related papers  (2021-06-04T17:54:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.