Neural Mesh-Based Graphics

• URL: http://arxiv.org/abs/2208.05785v1
• Date: Wed, 10 Aug 2022 09:18:28 GMT
• Title: Neural Mesh-Based Graphics
• Authors: Shubhendu Jena, Franck Multon, Adnane Boukhayma
• Abstract summary: We revisit NPBG, the popular approach to novel view synthesis that introduced the ubiquitous point feature neural paradigm. We achieve this through a view mesh-based point descriptorization, in addition to a foreground/background scene rendering split, and an improved loss. We also perform competitively with respect to the state-of-the-art method SVS, which has been trained on the full dataset.
• Score: 5.865500664175491
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We revisit NPBG, the popular approach to novel view synthesis that introduced the ubiquitous point feature neural rendering paradigm. We are interested in particular in data-efficient learning with fast view synthesis. We achieve this through a view-dependent mesh-based denser point descriptor rasterization, in addition to a foreground/background scene rendering split, and an improved loss. By training solely on a single scene, we outperform NPBG, which has been trained on ScanNet and then scene finetuned. We also perform competitively with respect to the state-of-the-art method SVS, which has been trained on the full dataset (DTU and Tanks and Temples) and then scene finetuned, in spite of their deeper neural renderer.

Related papers

• D-NPC: Dynamic Neural Point Clouds for Non-Rigid View Synthesis from Monocular Video [53.83936023443193]
  This paper contributes to the field by introducing a new synthesis method for dynamic novel view from monocular video, such as smartphone captures. Our approach represents the as a $textitdynamic neural point cloud$, an implicit time-conditioned point cloud that encodes local geometry and appearance in separate hash-encoded neural feature grids.
  arXiv  Detail & Related papers  (2024-06-14T14:35:44Z)
• DNS SLAM: Dense Neural Semantic-Informed SLAM [92.39687553022605]
  DNS SLAM is a novel neural RGB-D semantic SLAM approach featuring a hybrid representation. Our method integrates multi-view geometry constraints with image-based feature extraction to improve appearance details. Our experimental results achieve state-of-the-art performance on both synthetic data and real-world data tracking.
  arXiv  Detail & Related papers  (2023-11-30T21:34:44Z)
• 3D Reconstruction with Generalizable Neural Fields using Scene Priors [71.37871576124789]
  We introduce training generalizable Neural Fields incorporating scene Priors (NFPs) The NFP network maps any single-view RGB-D image into signed distance and radiance values. A complete scene can be reconstructed by merging individual frames in the volumetric space WITHOUT a fusion module.
  arXiv  Detail & Related papers  (2023-09-26T18:01:02Z)
• View Synthesis with Sculpted Neural Points [64.40344086212279]
  Implicit neural representations have achieved impressive visual quality but have drawbacks in computational efficiency. We propose a new approach that performs view synthesis using point clouds. It is the first point-based method to achieve better visual quality than NeRF while being more than 100x faster in rendering speed.
  arXiv  Detail & Related papers  (2022-05-12T03:54:35Z)
• NPBG++: Accelerating Neural Point-Based Graphics [14.366073496519139]
  NPBG++ is a novel view synthesis (NVS) task that achieves high rendering realism with low scene fitting time. Our method efficiently leverages the multiview observations and the point cloud of a static scene to predict a neural descriptor for each point. In our comparisons, the proposed system outperforms previous NVS approaches in terms of fitting and rendering runtimes while producing images of similar quality.
  arXiv  Detail & Related papers  (2022-03-24T19:59:39Z)
• Point-Based Neural Rendering with Per-View Optimization [5.306819482496464]
  We introduce a general approach that is with MVS, but allows further optimization of scene properties in the space of input views. A key element of our approach is our new differentiable point-based pipeline. We use these elements together in our neural splatting, that outperforms all previous methods both in quality and speed in almost all scenes we tested.
  arXiv  Detail & Related papers  (2021-09-06T11:19:31Z)
• Neural Rays for Occlusion-aware Image-based Rendering [108.34004858785896]
  We present a new neural representation, called Neural Ray (NeuRay), for the novel view synthesis (NVS) task with multi-view images as input. NeuRay can quickly generate high-quality novel view rendering images of unseen scenes with little finetuning.
  arXiv  Detail & Related papers  (2021-07-28T15:09:40Z)
• TRANSPR: Transparency Ray-Accumulating Neural 3D Scene Point Renderer [6.320273914694594]
  We propose and evaluate a neural point-based graphics method that can model semi-transparent scene parts. We show that novel views of semi-transparent point cloud scenes can be generated after training with our approach.
  arXiv  Detail & Related papers  (2020-09-06T21:19:18Z)
• Free View Synthesis [100.86844680362196]
  We present a method for novel view synthesis from input images that are freely distributed around a scene. Our method does not rely on a regular arrangement of input views, can synthesize images for free camera movement through the scene, and works for general scenes with unconstrained geometric layouts.
  arXiv  Detail & Related papers  (2020-08-12T18:16:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.