A Simple and Scalable Shape Representation for 3D Reconstruction

• URL: http://arxiv.org/abs/2005.04623v1
• Date: Sun, 10 May 2020 10:22:50 GMT
• Title: A Simple and Scalable Shape Representation for 3D Reconstruction
• Authors: Mateusz Michalkiewicz, Eugene Belilovsky, Mahsa Baktashmotlagh, Anders Eriksson
• Abstract summary: We show that we can obtain high quality 3D reconstruction using a linear decoder, obtained from principal component analysis on the signed distance function (SDF) of the surface. This approach allows easily scaling to larger resolutions.
• Score: 22.826897662839357
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning applied to the reconstruction of 3D shapes has seen growing interest. A popular approach to 3D reconstruction and generation in recent years has been the CNN encoder-decoder model usually applied in voxel space. However, this often scales very poorly with the resolution limiting the effectiveness of these models. Several sophisticated alternatives for decoding to 3D shapes have been proposed typically relying on complex deep learning architectures for the decoder model. In this work, we show that this additional complexity is not necessary, and that we can actually obtain high quality 3D reconstruction using a linear decoder, obtained from principal component analysis on the signed distance function (SDF) of the surface. This approach allows easily scaling to larger resolutions. We show in multiple experiments that our approach is competitive with state-of-the-art methods. It also allows the decoder to be fine-tuned on the target task using a loss designed specifically for SDF transforms, obtaining further gains.

Related papers

• GeoLRM: Geometry-Aware Large Reconstruction Model for High-Quality 3D Gaussian Generation [65.33726478659304]
  We introduce the Geometry-Aware Large Reconstruction Model (GeoLRM), an approach which can predict high-quality assets with 512k Gaussians and 21 input images in only 11 GB GPU memory. Previous works neglect the inherent sparsity of 3D structure and do not utilize explicit geometric relationships between 3D and 2D images. GeoLRM tackles these issues by incorporating a novel 3D-aware transformer structure that directly processes 3D points and uses deformable cross-attention mechanisms.
  arXiv  Detail & Related papers  (2024-06-21T17:49:31Z)
• TIFu: Tri-directional Implicit Function for High-Fidelity 3D Character Reconstruction [35.299242563565315]
  Tri-directional Implicit Function (TIFu) is a vector-level representation that increases global 3D consistencies while significantly reducing memory usage. We introduce a new algorithm in 3D reconstruction at an arbitrary resolution by aggregating vectors along three axes. Our approach achieves state-of-the-art performances in both our self-curated character dataset and the benchmark 3D human dataset.
  arXiv  Detail & Related papers  (2024-01-25T23:30:51Z)
• FineRecon: Depth-aware Feed-forward Network for Detailed 3D Reconstruction [13.157400338544177]
  Recent works on 3D reconstruction from posed images have demonstrated that direct inference of scene-level 3D geometry is feasible using deep neural networks. We propose three effective solutions for improving the fidelity of inference-based 3D reconstructions. Our method, FineRecon, produces smooth and highly accurate reconstructions, showing significant improvements across multiple depth and 3D reconstruction metrics.
  arXiv  Detail & Related papers  (2023-04-04T02:50:29Z)
• Make Encoder Great Again in 3D GAN Inversion through Geometry and Occlusion-Aware Encoding [25.86312557482366]
  3D GAN inversion aims to achieve high reconstruction fidelity and reasonable 3D geometry simultaneously from a single image input. We introduce a novel encoder-based inversion framework based on EG3D, one of the most widely-used 3D GAN models. Our method achieves impressive results comparable to optimization-based methods while operating up to 500 times faster.
  arXiv  Detail & Related papers  (2023-03-22T05:51:53Z)
• FIRe: Fast Inverse Rendering using Directional and Signed Distance Functions [97.5540646069663]
  We introduce a novel neural scene representation that we call the directional distance function (DDF) Our DDF is defined on the unit sphere and predicts the distance to the surface along any given direction. Based on our DDF, we present a novel fast algorithm (FIRe) to reconstruct 3D shapes given a posed depth map.
  arXiv  Detail & Related papers  (2022-03-30T13:24:04Z)
• 3D Shapes Local Geometry Codes Learning with SDF [8.37542758486152]
  A signed distance function (SDF) as the 3D shape description is one of the most effective approaches to represent 3D geometry for rendering and reconstruction. In this paper, we consider the degeneration problem of reconstruction coming from the capacity decrease of the DeepSDF model. We propose Local Geometry Code Learning (LGCL), a model that improves the original DeepSDF results by learning from a local shape geometry.
  arXiv  Detail & Related papers  (2021-08-19T09:56:03Z)
• Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes [77.6741486264257]
  We introduce an efficient neural representation that, for the first time, enables real-time rendering of high-fidelity neural SDFs. We show that our representation is 2-3 orders of magnitude more efficient in terms of rendering speed compared to previous works.
  arXiv  Detail & Related papers  (2021-01-26T18:50:22Z)
• Exploring Deep 3D Spatial Encodings for Large-Scale 3D Scene Understanding [19.134536179555102]
  We propose an alternative approach to overcome the limitations of CNN based approaches by encoding the spatial features of raw 3D point clouds into undirected graph models. The proposed method achieves on par state-of-the-art accuracy with improved training time and model stability thus indicating strong potential for further research.
  arXiv  Detail & Related papers  (2020-11-29T12:56:19Z)
• Learning Deformable Tetrahedral Meshes for 3D Reconstruction [78.0514377738632]
  3D shape representations that accommodate learning-based 3D reconstruction are an open problem in machine learning and computer graphics. Previous work on neural 3D reconstruction demonstrated benefits, but also limitations, of point cloud, voxel, surface mesh, and implicit function representations. We introduce Deformable Tetrahedral Meshes (DefTet) as a particular parameterization that utilizes volumetric tetrahedral meshes for the reconstruction problem.
  arXiv  Detail & Related papers  (2020-11-03T02:57:01Z)
• Convolutional Occupancy Networks [88.48287716452002]
  We propose Convolutional Occupancy Networks, a more flexible implicit representation for detailed reconstruction of objects and 3D scenes. By combining convolutional encoders with implicit occupancy decoders, our model incorporates inductive biases, enabling structured reasoning in 3D space. We empirically find that our method enables the fine-grained implicit 3D reconstruction of single objects, scales to large indoor scenes, and generalizes well from synthetic to real data.
  arXiv  Detail & Related papers  (2020-03-10T10:17:07Z)
• Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion [53.885984328273686]
  Implicit Feature Networks (IF-Nets) deliver continuous outputs, can handle multiple topologies, and complete shapes for missing or sparse input data. IF-Nets clearly outperform prior work in 3D object reconstruction in ShapeNet, and obtain significantly more accurate 3D human reconstructions.
  arXiv  Detail & Related papers  (2020-03-03T11:14:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.