Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images

• URL: http://arxiv.org/abs/2008.07760v1
• Date: Tue, 18 Aug 2020 06:33:40 GMT
• Title: Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images
• Authors: Jiahui Lei, Srinath Sridhar, Paul Guerrero, Minhyuk Sung, Niloy Mitra, Leonidas J. Guibas
• Abstract summary: We investigate the problem of learning to generate 3D parametric surface representations for novel object instances, as seen from one or more views. We design neural networks capable of generating high-quality parametric 3D surfaces which are consistent between views. Our method is supervised and trained on a public dataset of shapes from common object categories.
• Score: 64.53227129573293
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We investigate the problem of learning to generate 3D parametric surface representations for novel object instances, as seen from one or more views. Previous work on learning shape reconstruction from multiple views uses discrete representations such as point clouds or voxels, while continuous surface generation approaches lack multi-view consistency. We address these issues by designing neural networks capable of generating high-quality parametric 3D surfaces which are also consistent between views. Furthermore, the generated 3D surfaces preserve accurate image pixel to 3D surface point correspondences, allowing us to lift texture information to reconstruct shapes with rich geometry and appearance. Our method is supervised and trained on a public dataset of shapes from common object categories. Quantitative results indicate that our method significantly outperforms previous work, while qualitative results demonstrate the high quality of our reconstructions.

Related papers

• Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement [78.48648360358193]
  We present a novel framework that generates textured surface meshes from images. Our approach begins by efficiently initializing the geometry and view-dependency appearance with a NeRF. We jointly refine the appearance with geometry and bake it into texture images for real-time rendering.
  arXiv  Detail & Related papers  (2023-03-03T17:14:44Z)
• 3D Surface Reconstruction in the Wild by Deforming Shape Priors from Synthetic Data [24.97027425606138]
  Reconstructing the underlying 3D surface of an object from a single image is a challenging problem. We present a new method for joint category-specific 3D reconstruction and object pose estimation from a single image. Our approach achieves state-of-the-art reconstruction performance across several real-world datasets.
  arXiv  Detail & Related papers  (2023-02-24T20:37:27Z)
• Learning Neural Implicit Representations with Surface Signal Parameterizations [14.835882967340968]
  We present a neural network architecture that implicitly encodes the underlying surface parameterization suitable for appearance data. Our model remains compatible with existing mesh-based digital content with appearance data.
  arXiv  Detail & Related papers  (2022-11-01T15:10:58Z)
• Single-view 3D Mesh Reconstruction for Seen and Unseen Categories [69.29406107513621]
  Single-view 3D Mesh Reconstruction is a fundamental computer vision task that aims at recovering 3D shapes from single-view RGB images. This paper tackles Single-view 3D Mesh Reconstruction, to study the model generalization on unseen categories. We propose an end-to-end two-stage network, GenMesh, to break the category boundaries in reconstruction.
  arXiv  Detail & Related papers  (2022-08-04T14:13:35Z)
• Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Images [82.32776379815712]
  We study the problem of shape generation in 3D mesh representation from a small number of color images with or without camera poses. We adopt to further improve the shape quality by leveraging cross-view information with a graph convolution network. Our model is robust to the quality of the initial mesh and the error of camera pose, and can be combined with a differentiable function for test-time optimization.
  arXiv  Detail & Related papers  (2022-04-21T03:42:31Z)
• Beyond 3DMM: Learning to Capture High-fidelity 3D Face Shape [77.95154911528365]
  3D Morphable Model (3DMM) fitting has widely benefited face analysis due to its strong 3D priori. Previous reconstructed 3D faces suffer from degraded visual verisimilitude due to the loss of fine-grained geometry. This paper proposes a complete solution to capture the personalized shape so that the reconstructed shape looks identical to the corresponding person.
  arXiv  Detail & Related papers  (2022-04-09T03:46:18Z)
• Texturify: Generating Textures on 3D Shape Surfaces [34.726179801982646]
  We propose Texturify to learn a 3D shape that predicts texture on the 3D input. Our method does not require any 3D color supervision to learn 3D objects.
  arXiv  Detail & Related papers  (2022-04-05T18:00:04Z)
• Implicit Neural Deformation for Multi-View Face Reconstruction [43.88676778013593]
  We present a new method for 3D face reconstruction from multi-view RGB images. Unlike previous methods which are built upon 3D morphable models, our method leverages an implicit representation to encode rich geometric features. Our experimental results on several benchmark datasets demonstrate that our approach outperforms alternative baselines and achieves superior face reconstruction results compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-12-05T07:02:53Z)
• H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction [27.66008315400462]
  Recent learning approaches that implicitly represent surface geometry have shown impressive results in the problem of multi-view 3D reconstruction. We tackle these limitations for the specific problem of few-shot full 3D head reconstruction. We learn a shape model of 3D heads from thousands of incomplete raw scans using implicit representations.
  arXiv  Detail & Related papers  (2021-07-26T23:04:18Z)
• Learning 3D Face Reconstruction with a Pose Guidance Network [49.13404714366933]
  We present a self-supervised learning approach to learning monocular 3D face reconstruction with a pose guidance network (PGN) First, we unveil the bottleneck of pose estimation in prior parametric 3D face learning methods, and propose to utilize 3D face landmarks for estimating pose parameters. With our specially designed PGN, our model can learn from both faces with fully labeled 3D landmarks and unlimited unlabeled in-the-wild face images.
  arXiv  Detail & Related papers  (2020-10-09T06:11:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.