D-OccNet: Detailed 3D Reconstruction Using Cross-Domain Learning

• URL: http://arxiv.org/abs/2104.13854v1
• Date: Wed, 28 Apr 2021 16:00:54 GMT
• Title: D-OccNet: Detailed 3D Reconstruction Using Cross-Domain Learning
• Authors: Minhaj Uddin Ansari, Talha Bilal, Naeem Akhter
• Abstract summary: We extend the work on Occupancy Networks by exploiting cross-domain learning of image and point cloud domains. Our network, the Double Occupancy Network (D-OccNet) outperforms Occupancy Networks in terms of visual quality and details captured in the 3D reconstruction.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning based 3D reconstruction of single view 2D image is becoming increasingly popular due to their wide range of real-world applications, but this task is inherently challenging because of the partial observability of an object from a single perspective. Recently, state of the art probability based Occupancy Networks reconstructed 3D surfaces from three different types of input domains: single view 2D image, point cloud and voxel. In this study, we extend the work on Occupancy Networks by exploiting cross-domain learning of image and point cloud domains. Specifically, we first convert the single view 2D image into a simpler point cloud representation, and then reconstruct a 3D surface from it. Our network, the Double Occupancy Network (D-OccNet) outperforms Occupancy Networks in terms of visual quality and details captured in the 3D reconstruction.

Related papers

• DI-Net : Decomposed Implicit Garment Transfer Network for Digital Clothed 3D Human [75.45488434002898]
  Existing 2D virtual try-on methods cannot be directly extended to 3D since they lack the ability to perceive the depth of each pixel. We propose a Decomposed Implicit garment transfer network (DI-Net), which can effortlessly reconstruct a 3D human mesh with the newly try-on result.
  arXiv  Detail & Related papers  (2023-11-28T14:28:41Z)
• Mask3D: Pre-training 2D Vision Transformers by Learning Masked 3D Priors [29.419069066603438]
  We propose Mask3D to leverage existing large-scale RGB-D data in a self-supervised pre-training to embed 3D priors into 2D learned feature representations. We demonstrate Mask3D is particularly effective in embedding 3D priors into the powerful 2D ViT backbone, enabling improved representation learning for various scene understanding tasks.
  arXiv  Detail & Related papers  (2023-02-28T16:45:21Z)
• Voxel-based 3D Detection and Reconstruction of Multiple Objects from a Single Image [22.037472446683765]
  We learn a regular grid of 3D voxel features from the input image which is aligned with 3D scene space via a 3D feature lifting operator. Based on the 3D voxel features, our novel CenterNet-3D detection head formulates the 3D detection as keypoint detection in the 3D space. We devise an efficient coarse-to-fine reconstruction module, including coarse-level voxelization and a novel local PCA-SDF shape representation.
  arXiv  Detail & Related papers  (2021-11-04T18:30:37Z)
• Bidirectional Projection Network for Cross Dimension Scene Understanding [69.29443390126805]
  We present a emphbidirectional projection network (BPNet) for joint 2D and 3D reasoning in an end-to-end manner. Via the emphBPM, complementary 2D and 3D information can interact with each other in multiple architectural levels. Our emphBPNet achieves top performance on the ScanNetV2 benchmark for both 2D and 3D semantic segmentation.
  arXiv  Detail & Related papers  (2021-03-26T08:31:39Z)
• Learning Joint 2D-3D Representations for Depth Completion [90.62843376586216]
  We design a simple yet effective neural network block that learns to extract joint 2D and 3D features. Specifically, the block consists of two domain-specific sub-networks that apply 2D convolution on image pixels and continuous convolution on 3D points.
  arXiv  Detail & Related papers  (2020-12-22T22:58:29Z)
• ParaNet: Deep Regular Representation for 3D Point Clouds [62.81379889095186]
  ParaNet is a novel end-to-end deep learning framework for representing 3D point clouds. It converts an irregular 3D point cloud into a regular 2D color image, named point geometry image (PGI) In contrast to conventional regular representation modalities based on multi-view projection and voxelization, the proposed representation is differentiable and reversible.
  arXiv  Detail & Related papers  (2020-12-05T13:19:55Z)
• GRF: Learning a General Radiance Field for 3D Representation and Rendering [4.709764624933227]
  We present a simple yet powerful neural network that implicitly represents and renders 3D objects and scenes only from 2D observations. The network models 3D geometries as a general radiance field, which takes a set of 2D images with camera poses and intrinsics as input. Our method can generate high-quality and realistic novel views for novel objects, unseen categories and challenging real-world scenes.
  arXiv  Detail & Related papers  (2020-10-09T14:21:43Z)
• Improved Modeling of 3D Shapes with Multi-view Depth Maps [48.8309897766904]
  We present a general-purpose framework for modeling 3D shapes using CNNs. Using just a single depth image of the object, we can output a dense multi-view depth map representation of 3D objects.
  arXiv  Detail & Related papers  (2020-09-07T17:58:27Z)
• From Image Collections to Point Clouds with Self-supervised Shape and Pose Networks [53.71440550507745]
  Reconstructing 3D models from 2D images is one of the fundamental problems in computer vision. We propose a deep learning technique for 3D object reconstruction from a single image. We learn both 3D point cloud reconstruction and pose estimation networks in a self-supervised manner.
  arXiv  Detail & Related papers  (2020-05-05T04:25:16Z)
• DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction [24.903382114775283]
  Reconstructing 3D shapes from single-view images has been a long-standing research problem. We present DISN, a Deep Implicit Surface Network which can generate a high-quality detail-rich 3D mesh from an 2D image. To the best of our knowledge, DISN is the first method that constantly captures details such as holes and thin structures present in 3D shapes from single-view images.
  arXiv  Detail & Related papers  (2019-05-26T01:58:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.