SparseCraft: Few-Shot Neural Reconstruction through Stereopsis Guided Geometric Linearization

• URL: http://arxiv.org/abs/2407.14257v1
• Date: Fri, 19 Jul 2024 12:36:36 GMT
• Title: SparseCraft: Few-Shot Neural Reconstruction through Stereopsis Guided Geometric Linearization
• Authors: Mae Younes, Amine Ouasfi, Adnane Boukhayma,
• Abstract summary: We present a novel approach for recovering 3D shape and view dependent appearance from a few colored images. Our method learns an implicit neural representation in the form of a Signed Distance Function (SDF) and a radiance field. Key to our contribution is a novel implicit neural shape function learning strategy that encourages our SDF field to be as linear as possible near the level-set.
• Score: 7.769607568805291
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a novel approach for recovering 3D shape and view dependent appearance from a few colored images, enabling efficient 3D reconstruction and novel view synthesis. Our method learns an implicit neural representation in the form of a Signed Distance Function (SDF) and a radiance field. The model is trained progressively through ray marching enabled volumetric rendering, and regularized with learning-free multi-view stereo (MVS) cues. Key to our contribution is a novel implicit neural shape function learning strategy that encourages our SDF field to be as linear as possible near the level-set, hence robustifying the training against noise emanating from the supervision and regularization signals. Without using any pretrained priors, our method, called SparseCraft, achieves state-of-the-art performances both in novel-view synthesis and reconstruction from sparse views in standard benchmarks, while requiring less than 10 minutes for training.

Related papers

• FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training [15.634646420318731]
  We present a 3D Gaussian-based novel view synthesis method using sparse input images. We propose a multi-stage training scheme with matching-based consistency constraints imposed on the novel views. This is achieved by using the matches of the available training images to supervise the generation of the novel views.
  arXiv  Detail & Related papers  (2024-11-04T16:21:00Z)
• Few-Shot Unsupervised Implicit Neural Shape Representation Learning with Spatial Adversaries [8.732260277121547]
  Implicit Neural Representations have gained prominence as a powerful framework for capturing complex data modalities. Within the realm of 3D shape representation, Neural Signed Distance Functions (SDF) have demonstrated remarkable potential in faithfully encoding intricate shape geometry.
  arXiv  Detail & Related papers  (2024-08-27T14:54:33Z)
• DNS SLAM: Dense Neural Semantic-Informed SLAM [92.39687553022605]
  DNS SLAM is a novel neural RGB-D semantic SLAM approach featuring a hybrid representation. Our method integrates multi-view geometry constraints with image-based feature extraction to improve appearance details. Our experimental results achieve state-of-the-art performance on both synthetic data and real-world data tracking.
  arXiv  Detail & Related papers  (2023-11-30T21:34:44Z)
• Learning to Render Novel Views from Wide-Baseline Stereo Pairs [26.528667940013598]
  We introduce a method for novel view synthesis given only a single wide-baseline stereo image pair. Existing approaches to novel view synthesis from sparse observations fail due to recovering incorrect 3D geometry. We propose an efficient, image-space epipolar line sampling scheme to assemble image features for a target ray.
  arXiv  Detail & Related papers  (2023-04-17T17:40:52Z)
• Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction [77.69363640021503]
  3D-aware image synthesis encompasses a variety of tasks, such as scene generation and novel view synthesis from images. We present SSDNeRF, a unified approach that employs an expressive diffusion model to learn a generalizable prior of neural radiance fields (NeRF) from multi-view images of diverse objects.
  arXiv  Detail & Related papers  (2023-04-13T17:59:01Z)
• SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fields [19.740018132105757]
  SceneRF is a self-supervised monocular scene reconstruction method using only posed image sequences for training. At inference, a single input image suffices to hallucinate novel depth views which are fused together to obtain 3D scene reconstruction.
  arXiv  Detail & Related papers  (2022-12-05T18:59:57Z)
• Neural 3D Reconstruction in the Wild [86.6264706256377]
  We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections. We present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes.
  arXiv  Detail & Related papers  (2022-05-25T17:59:53Z)
• Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering [60.02806355570514]
  Inferring representations of 3D scenes from 2D observations is a fundamental problem of computer graphics, computer vision, and artificial intelligence. We propose a novel neural scene representation, Light Field Networks or LFNs, which represent both geometry and appearance of the underlying 3D scene in a 360-degree, four-dimensional light field. Rendering a ray from an LFN requires only a *single* network evaluation, as opposed to hundreds of evaluations per ray for ray-marching or based on volumetrics.
  arXiv  Detail & Related papers  (2021-06-04T17:54:49Z)
• MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo [52.329580781898116]
  We present MVSNeRF, a novel neural rendering approach that can efficiently reconstruct neural radiance fields for view synthesis. Unlike prior works on neural radiance fields that consider per-scene optimization on densely captured images, we propose a generic deep neural network that can reconstruct radiance fields from only three nearby input views via fast network inference.
  arXiv  Detail & Related papers  (2021-03-29T13:15:23Z)
• SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images [44.78174845839193]
  Recent efforts have turned to learning 3D reconstruction without 3D supervision from RGB images with annotated 2D silhouettes. These techniques still require multi-view annotations of the same object instance during training. We propose SDF-SRN, an approach that requires only a single view of objects at training time.
  arXiv  Detail & Related papers  (2020-10-20T17:59:47Z)
• Neural Descent for Visual 3D Human Pose and Shape [67.01050349629053]
  We present deep neural network methodology to reconstruct the 3d pose and shape of people, given an input RGB image. We rely on a recently introduced, expressivefull body statistical 3d human model, GHUM, trained end-to-end. Central to our methodology, is a learning to learn and optimize approach, referred to as HUmanNeural Descent (HUND), which avoids both second-order differentiation.
  arXiv  Detail & Related papers  (2020-08-16T13:38:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.