Probabilistic Implicit Scene Completion

• URL: http://arxiv.org/abs/2204.01264v1
• Date: Mon, 4 Apr 2022 06:16:54 GMT
• Title: Probabilistic Implicit Scene Completion
• Authors: Dongsu Zhang, Changwoon Choi, Inbum Park, Young Min Kim
• Abstract summary: We propose a probabilistic shape completion method extended to the continuous geometry of large-scale 3D scenes. We employ the Generative Cellular Automata that learns the multi-modal distribution and transform the formulation to process large-scale continuous geometry. Experiments show that our model successfully generates diverse plausible scenes faithful to the input, especially when the input suffers from a significant amount of missing data.
• Score: 6.954686339092988
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a probabilistic shape completion method extended to the continuous geometry of large-scale 3D scenes. Real-world scans of 3D scenes suffer from a considerable amount of missing data cluttered with unsegmented objects. The problem of shape completion is inherently ill-posed, and high-quality result requires scalable solutions that consider multiple possible outcomes. We employ the Generative Cellular Automata that learns the multi-modal distribution and transform the formulation to process large-scale continuous geometry. The local continuous shape is incrementally generated as a sparse voxel embedding, which contains the latent code for each occupied cell. We formally derive that our training objective for the sparse voxel embedding maximizes the variational lower bound of the complete shape distribution and therefore our progressive generation constitutes a valid generative model. Experiments show that our model successfully generates diverse plausible scenes faithful to the input, especially when the input suffers from a significant amount of missing data. We also demonstrate that our approach outperforms deterministic models even in less ambiguous cases with a small amount of missing data, which infers that probabilistic formulation is crucial for high-quality geometry completion on input scans exhibiting any levels of completeness.

Related papers

• Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding [50.448520056844885]
  We propose a generative Bayesian network to produce diverse synthetic scenes with real-world patterns. A series of experiments robustly display our method's consistent superiority over existing state-of-the-art pre-training approaches.
  arXiv  Detail & Related papers  (2024-06-17T07:43:53Z)
• Diverse Shape Completion via Style Modulated Generative Adversarial Networks [0.0]
  Shape completion aims to recover the full 3D geometry of an object from a partial observation. This problem is inherently multi-modal since there can be many ways to plausibly complete the missing regions of a shape. We propose a novel conditional generative adversarial network that can produce many diverse plausible completions of a partially observed point cloud.
  arXiv  Detail & Related papers  (2023-11-18T23:40:20Z)
• 3D Adversarial Augmentations for Robust Out-of-Domain Predictions [115.74319739738571]
  We focus on improving the generalization to out-of-domain data. We learn a set of vectors that deform the objects in an adversarial fashion. We perform adversarial augmentation by applying the learned sample-independent vectors to the available objects when training a model.
  arXiv  Detail & Related papers  (2023-08-29T17:58:55Z)
• DiffComplete: Diffusion-based Generative 3D Shape Completion [114.43353365917015]
  We introduce a new diffusion-based approach for shape completion on 3D range scans. We strike a balance between realism, multi-modality, and high fidelity. DiffComplete sets a new SOTA performance on two large-scale 3D shape completion benchmarks.
  arXiv  Detail & Related papers  (2023-06-28T16:07:36Z)
• T1: Scaling Diffusion Probabilistic Fields to High-Resolution on Unified Visual Modalities [69.16656086708291]
  Diffusion Probabilistic Field (DPF) models the distribution of continuous functions defined over metric spaces. We propose a new model comprising of a view-wise sampling algorithm to focus on local structure learning. The model can be scaled to generate high-resolution data while unifying multiple modalities.
  arXiv  Detail & Related papers  (2023-05-24T03:32:03Z)
• Probabilistic Registration for Gaussian Process 3D shape modelling in the presence of extensive missing data [63.8376359764052]
  We propose a shape fitting/registration method based on a Gaussian Processes formulation, suitable for shapes with extensive regions of missing data. Experiments are conducted both for a 2D small dataset with diverse transformations and a 3D dataset of ears.
  arXiv  Detail & Related papers  (2022-03-26T16:48:27Z)
• ShapeFormer: Transformer-based Shape Completion via Sparse Representation [41.33457875133559]
  We present ShapeFormer, a network that produces a distribution of object completions conditioned on incomplete, and possibly noisy, point clouds. The resultant distribution can then be sampled to generate likely completions, each exhibiting plausible shape details while being faithful to the input.
  arXiv  Detail & Related papers  (2022-01-25T13:58:30Z)
• 3D Shape Generation and Completion through Point-Voxel Diffusion [24.824065748889048]
  We propose a novel approach for probabilistic generative modeling of 3D shapes. Point-Voxel Diffusion (PVD) is a unified, probabilistic formulation for unconditional shape generation and conditional, multimodal shape completion. PVD can be viewed as a series of denoising steps, reversing the diffusion process from observed point cloud data to Gaussian noise, and is trained by optimizing a variational lower bound to the (conditional) likelihood function.
  arXiv  Detail & Related papers  (2021-04-08T10:38:03Z)
• Dense Non-Rigid Structure from Motion: A Manifold Viewpoint [162.88686222340962]
  Non-Rigid Structure-from-Motion (NRSfM) problem aims to recover 3D geometry of a deforming object from its 2D feature correspondences across multiple frames. We show that our approach significantly improves accuracy, scalability, and robustness against noise.
  arXiv  Detail & Related papers  (2020-06-15T09:15:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.