DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction

• URL: http://arxiv.org/abs/1905.10711v5
• Date: Mon, 25 Mar 2024 22:10:45 GMT
• Title: DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction
• Authors: Qiangeng Xu, Weiyue Wang, Duygu Ceylan, Radomir Mech, Ulrich Neumann,
• Abstract summary: Reconstructing 3D shapes from single-view images has been a long-standing research problem. We present DISN, a Deep Implicit Surface Network which can generate a high-quality detail-rich 3D mesh from an 2D image. To the best of our knowledge, DISN is the first method that constantly captures details such as holes and thin structures present in 3D shapes from single-view images.
• Score: 24.903382114775283
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reconstructing 3D shapes from single-view images has been a long-standing research problem. In this paper, we present DISN, a Deep Implicit Surface Network which can generate a high-quality detail-rich 3D mesh from an 2D image by predicting the underlying signed distance fields. In addition to utilizing global image features, DISN predicts the projected location for each 3D point on the 2D image, and extracts local features from the image feature maps. Combining global and local features significantly improves the accuracy of the signed distance field prediction, especially for the detail-rich areas. To the best of our knowledge, DISN is the first method that constantly captures details such as holes and thin structures present in 3D shapes from single-view images. DISN achieves the state-of-the-art single-view reconstruction performance on a variety of shape categories reconstructed from both synthetic and real images. Code is available at https://github.com/xharlie/DISN The supplementary can be found at https://xharlie.github.io/images/neurips_2019_supp.pdf

Related papers

• Vision Transformer for NeRF-Based View Synthesis from a Single Input Image [49.956005709863355]
  We propose to leverage both the global and local features to form an expressive 3D representation. To synthesize a novel view, we train a multilayer perceptron (MLP) network conditioned on the learned 3D representation to perform volume rendering. Our method can render novel views from only a single input image and generalize across multiple object categories using a single model.
  arXiv  Detail & Related papers  (2022-07-12T17:52:04Z)
• D-OccNet: Detailed 3D Reconstruction Using Cross-Domain Learning [0.0]
  We extend the work on Occupancy Networks by exploiting cross-domain learning of image and point cloud domains. Our network, the Double Occupancy Network (D-OccNet) outperforms Occupancy Networks in terms of visual quality and details captured in the 3D reconstruction.
  arXiv  Detail & Related papers  (2021-04-28T16:00:54Z)
• Bidirectional Projection Network for Cross Dimension Scene Understanding [69.29443390126805]
  We present a emphbidirectional projection network (BPNet) for joint 2D and 3D reasoning in an end-to-end manner. Via the emphBPM, complementary 2D and 3D information can interact with each other in multiple architectural levels. Our emphBPNet achieves top performance on the ScanNetV2 benchmark for both 2D and 3D semantic segmentation.
  arXiv  Detail & Related papers  (2021-03-26T08:31:39Z)
• Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes [77.6741486264257]
  We introduce an efficient neural representation that, for the first time, enables real-time rendering of high-fidelity neural SDFs. We show that our representation is 2-3 orders of magnitude more efficient in terms of rendering speed compared to previous works.
  arXiv  Detail & Related papers  (2021-01-26T18:50:22Z)
• D$^2$IM-Net: Learning Detail Disentangled Implicit Fields from Single Images [6.121310352120004]
  We present the first single-view 3D reconstruction network aimed at recovering geometric details from an input image. Our key idea is to train the network to learn a detail disentangled reconstruction consisting of two functions.
  arXiv  Detail & Related papers  (2020-12-11T21:42:52Z)
• GRF: Learning a General Radiance Field for 3D Representation and Rendering [4.709764624933227]
  We present a simple yet powerful neural network that implicitly represents and renders 3D objects and scenes only from 2D observations. The network models 3D geometries as a general radiance field, which takes a set of 2D images with camera poses and intrinsics as input. Our method can generate high-quality and realistic novel views for novel objects, unseen categories and challenging real-world scenes.
  arXiv  Detail & Related papers  (2020-10-09T14:21:43Z)
• Improved Modeling of 3D Shapes with Multi-view Depth Maps [48.8309897766904]
  We present a general-purpose framework for modeling 3D shapes using CNNs. Using just a single depth image of the object, we can output a dense multi-view depth map representation of 3D objects.
  arXiv  Detail & Related papers  (2020-09-07T17:58:27Z)
• Pix2Vox++: Multi-scale Context-aware 3D Object Reconstruction from Single and Multiple Images [56.652027072552606]
  We propose a novel framework for single-view and multi-view 3D object reconstruction, named Pix2Vox++. By using a well-designed encoder-decoder, it generates a coarse 3D volume from each input image. A multi-scale context-aware fusion module is then introduced to adaptively select high-quality reconstructions for different parts from all coarse 3D volumes to obtain a fused 3D volume.
  arXiv  Detail & Related papers  (2020-06-22T13:48:09Z)
• 3D Human Mesh Regression with Dense Correspondence [95.92326689172877]
  Estimating 3D mesh of the human body from a single 2D image is an important task with many applications such as augmented reality and Human-Robot interaction. Prior works reconstructed 3D mesh from global image feature extracted by using convolutional neural network (CNN), where the dense correspondences between the mesh surface and the image pixels are missing. This paper proposes a model-free 3D human mesh estimation framework, named DecoMR, which explicitly establishes the dense correspondence between the mesh and the local image features in the UV space.
  arXiv  Detail & Related papers  (2020-06-10T08:50:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.