Learning Pose-invariant 3D Object Reconstruction from Single-view Images

• URL: http://arxiv.org/abs/2004.01347v2
• Date: Mon, 27 Jul 2020 08:11:32 GMT
• Title: Learning Pose-invariant 3D Object Reconstruction from Single-view Images
• Authors: Bo Peng, Wei Wang, Jing Dong and Tieniu Tan
• Abstract summary: In this paper, we explore a more realistic setup of learning 3D shape from only single-view images. The major difficulty lies in insufficient constraints that can be provided by single view images. We propose an effective adversarial domain confusion method to learn pose-disentangled compact shape space.
• Score: 61.98279201609436
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning to reconstruct 3D shapes using 2D images is an active research topic, with benefits of not requiring expensive 3D data. However, most work in this direction requires multi-view images for each object instance as training supervision, which oftentimes does not apply in practice. In this paper, we relax the common multi-view assumption and explore a more challenging yet more realistic setup of learning 3D shape from only single-view images. The major difficulty lies in insufficient constraints that can be provided by single view images, which leads to the problem of pose entanglement in learned shape space. As a result, reconstructed shapes vary along input pose and have poor accuracy. We address this problem by taking a novel domain adaptation perspective, and propose an effective adversarial domain confusion method to learn pose-disentangled compact shape space. Experiments on single-view reconstruction show effectiveness in solving pose entanglement, and the proposed method achieves on-par reconstruction accuracy with state-of-the-art with higher efficiency.

Related papers

• MeTTA: Single-View to 3D Textured Mesh Reconstruction with Test-Time Adaptation [19.15982759396811]
  We propose MeTTA, a test-time adaptation exploiting generative prior. We design joint optimization of 3D geometry, appearance, and pose to handle OoD cases with only a single view image. MeTTA effectively deals with OoD scenarios at failure cases of existing learning-based 3D reconstruction models.
  arXiv  Detail & Related papers  (2024-08-21T09:35:16Z)
• Few-View Object Reconstruction with Unknown Categories and Camera Poses [80.0820650171476]
  This work explores reconstructing general real-world objects from a few images without known camera poses or object categories. The crux of our work is solving two fundamental 3D vision problems -- shape reconstruction and pose estimation. Our method FORGE predicts 3D features from each view and leverages them in conjunction with the input images to establish cross-view correspondence.
  arXiv  Detail & Related papers  (2022-12-08T18:59:02Z)
• 3D-Augmented Contrastive Knowledge Distillation for Image-based Object Pose Estimation [4.415086501328683]
  We deal with the problem in a reasonable new setting, namely 3D shape is exploited in the training process, and the testing is still purely image-based. We propose a novel contrastive knowledge distillation framework that effectively transfers 3D-augmented image representation from a multi-modal model to an image-based model. We experimentally report state-of-the-art results compared with existing category-agnostic image-based methods by a large margin.
  arXiv  Detail & Related papers  (2022-06-02T16:46:18Z)
• Unsupervised Severely Deformed Mesh Reconstruction (DMR) from a Single-View Image [26.464091507125826]
  We introduce a template-based method to infer 3D shapes from a single-view image and apply the reconstructed mesh to a downstream task. Our method faithfully reconstructs 3D meshes and achieves state-of-the-art accuracy in a length measurement task on a severely deformed fish dataset.
  arXiv  Detail & Related papers  (2022-01-23T21:46:30Z)
• Toward Realistic Single-View 3D Object Reconstruction with Unsupervised Learning from Multiple Images [18.888384816156744]
  We propose a novel unsupervised algorithm to learn a 3D reconstruction network from a multi-image dataset. Our algorithm is more general and covers the symmetry-required scenario as a special case. Our method surpasses the previous work in both quality and robustness.
  arXiv  Detail & Related papers  (2021-09-06T08:34:04Z)
• Discovering 3D Parts from Image Collections [98.16987919686709]
  We tackle the problem of 3D part discovery from only 2D image collections. Instead of relying on manually annotated parts for supervision, we propose a self-supervised approach. Our key insight is to learn a novel part shape prior that allows each part to fit an object shape faithfully while constrained to have simple geometry.
  arXiv  Detail & Related papers  (2021-07-28T20:29:16Z)
• Neural Articulated Radiance Field [90.91714894044253]
  We present Neural Articulated Radiance Field (NARF), a novel deformable 3D representation for articulated objects learned from images. Experiments show that the proposed method is efficient and can generalize well to novel poses.
  arXiv  Detail & Related papers  (2021-04-07T13:23:14Z)
• 3D Human Shape and Pose from a Single Low-Resolution Image with Self-Supervised Learning [105.49950571267715]
  Existing deep learning methods for 3D human shape and pose estimation rely on relatively high-resolution input images. We propose RSC-Net, which consists of a Resolution-aware network, a Self-supervision loss, and a Contrastive learning scheme. We show that both these new training losses provide robustness when learning 3D shape and pose in a weakly-supervised manner.
  arXiv  Detail & Related papers  (2020-07-27T16:19:52Z)
• From Image Collections to Point Clouds with Self-supervised Shape and Pose Networks [53.71440550507745]
  Reconstructing 3D models from 2D images is one of the fundamental problems in computer vision. We propose a deep learning technique for 3D object reconstruction from a single image. We learn both 3D point cloud reconstruction and pose estimation networks in a self-supervised manner.
  arXiv  Detail & Related papers  (2020-05-05T04:25:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.