Diverse Shape Completion via Style Modulated Generative Adversarial Networks

• URL: http://arxiv.org/abs/2311.11184v1
• Date: Sat, 18 Nov 2023 23:40:20 GMT
• Title: Diverse Shape Completion via Style Modulated Generative Adversarial Networks
• Authors: Wesley Khademi, Li Fuxin
• Abstract summary: Shape completion aims to recover the full 3D geometry of an object from a partial observation. This problem is inherently multi-modal since there can be many ways to plausibly complete the missing regions of a shape. We propose a novel conditional generative adversarial network that can produce many diverse plausible completions of a partially observed point cloud.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Shape completion aims to recover the full 3D geometry of an object from a partial observation. This problem is inherently multi-modal since there can be many ways to plausibly complete the missing regions of a shape. Such diversity would be indicative of the underlying uncertainty of the shape and could be preferable for downstream tasks such as planning. In this paper, we propose a novel conditional generative adversarial network that can produce many diverse plausible completions of a partially observed point cloud. To enable our network to produce multiple completions for the same partial input, we introduce stochasticity into our network via style modulation. By extracting style codes from complete shapes during training, and learning a distribution over them, our style codes can explicitly carry shape category information leading to better completions. We further introduce diversity penalties and discriminators at multiple scales to prevent conditional mode collapse and to train without the need for multiple ground truth completions for each partial input. Evaluations across several synthetic and real datasets demonstrate that our method achieves significant improvements in respecting the partial observations while obtaining greater diversity in completions.

Related papers

• DiffComplete: Diffusion-based Generative 3D Shape Completion [114.43353365917015]
  We introduce a new diffusion-based approach for shape completion on 3D range scans. We strike a balance between realism, multi-modality, and high fidelity. DiffComplete sets a new SOTA performance on two large-scale 3D shape completion benchmarks.
  arXiv  Detail & Related papers  (2023-06-28T16:07:36Z)
• PatchRD: Detail-Preserving Shape Completion by Learning Patch Retrieval and Deformation [59.70430570779819]
  We introduce a data-driven shape completion approach that focuses on completing geometric details of missing regions of 3D shapes. Our key insight is to copy and deform patches from the partial input to complete missing regions. We leverage repeating patterns by retrieving patches from the partial input, and learn global structural priors by using a neural network to guide the retrieval and deformation steps.
  arXiv  Detail & Related papers  (2022-07-24T18:59:09Z)
• Implicit Shape Completion via Adversarial Shape Priors [46.48590354256945]
  We present a novel neural implicit shape method for partial point cloud completion. We combine a conditional Deep-SDF architecture with learned, adversarial shape priors. We train a PointNet++ discriminator that impels the generator to produce plausible, globally consistent reconstructions.
  arXiv  Detail & Related papers  (2022-04-21T12:49:59Z)
• Probabilistic Implicit Scene Completion [6.954686339092988]
  We propose a probabilistic shape completion method extended to the continuous geometry of large-scale 3D scenes. We employ the Generative Cellular Automata that learns the multi-modal distribution and transform the formulation to process large-scale continuous geometry. Experiments show that our model successfully generates diverse plausible scenes faithful to the input, especially when the input suffers from a significant amount of missing data.
  arXiv  Detail & Related papers  (2022-04-04T06:16:54Z)
• Shape Completion via IMLE [9.716911810130576]
  Shape completion is the problem of completing partial input shapes such as partial scans. We propose a novel multimodal shape completion technique that is effectively able to learn a one-to-many mapping. We show that our method is superior to alternatives in terms of completeness and diversity of shapes.
  arXiv  Detail & Related papers  (2021-06-30T17:45:10Z)
• Unsupervised 3D Shape Completion through GAN Inversion [116.27680045885849]
  We present ShapeInversion, which introduces Generative Adrial Network (GAN) inversion to shape completion for the first time. ShapeInversion uses a GAN pre-trained on complete shapes by searching for a latent code that gives a complete shape that best fits the given partial input. On the ShapeNet benchmark, the proposed ShapeInversion outperforms the SOTA unsupervised method, and is comparable with supervised methods that are learned using paired data.
  arXiv  Detail & Related papers  (2021-04-27T17:53:46Z)
• Cascaded Refinement Network for Point Cloud Completion with Self-supervision [74.80746431691938]
  We introduce a two-branch network for shape completion. The first branch is a cascaded shape completion sub-network to synthesize complete objects. The second branch is an auto-encoder to reconstruct the original partial input.
  arXiv  Detail & Related papers  (2020-10-17T04:56:22Z)
• Weakly-supervised 3D Shape Completion in the Wild [91.04095516680438]
  We address the problem of learning 3D complete shape from unaligned and real-world partial point clouds. We propose a weakly-supervised method to estimate both 3D canonical shape and 6-DoF pose for alignment, given multiple partial observations. Experiments on both synthetic and real data show that it is feasible and promising to learn 3D shape completion through large-scale data without shape and pose supervision.
  arXiv  Detail & Related papers  (2020-08-20T17:53:42Z)
• Cascaded Refinement Network for Point Cloud Completion [74.80746431691938]
  We propose a cascaded refinement network together with a coarse-to-fine strategy to synthesize the detailed object shapes. Considering the local details of partial input with the global shape information together, we can preserve the existing details in the incomplete point set. We also design a patch discriminator that guarantees every local area has the same pattern with the ground truth to learn the complicated point distribution.
  arXiv  Detail & Related papers  (2020-04-07T13:03:29Z)
• Multimodal Shape Completion via Conditional Generative Adversarial Networks [34.51271516263473]
  Several deep learning methods have been proposed for completing partial data from shape acquisition setups. We seek to complete the partial shape with multiple outputs by learning a one-to-many mapping. We develop the first multimodal shape completion method that completes the partial shape via conditional generative modeling.
  arXiv  Detail & Related papers  (2020-03-17T13:37:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.