PRISM: Probabilistic Representation for Integrated Shape Modeling and Generation

• URL: http://arxiv.org/abs/2504.04454v1
• Date: Sun, 06 Apr 2025 11:48:08 GMT
• Title: PRISM: Probabilistic Representation for Integrated Shape Modeling and Generation
• Authors: Lei Cheng, Mahdi Saleh, Qing Cheng, Lu Sang, Hongli Xu, Daniel Cremers, Federico Tombari,
• Abstract summary: PRISM is a novel approach for 3D shape generation that integrates categorical diffusion models with Statistical Shape Models (SSM) and Gaussian Mixture Models (GMM)<n>Our method employs compositional SSMs to capture part-level geometric variations and uses GMM to represent part semantics in a continuous space.<n>Our approach significantly outperforms previous methods in both quality and controllability of part-level operations.
• Score: 79.46526296655776
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the advancements in 3D full-shape generation, accurately modeling complex geometries and semantics of shape parts remains a significant challenge, particularly for shapes with varying numbers of parts. Current methods struggle to effectively integrate the contextual and structural information of 3D shapes into their generative processes. We address these limitations with PRISM, a novel compositional approach for 3D shape generation that integrates categorical diffusion models with Statistical Shape Models (SSM) and Gaussian Mixture Models (GMM). Our method employs compositional SSMs to capture part-level geometric variations and uses GMM to represent part semantics in a continuous space. This integration enables both high fidelity and diversity in generated shapes while preserving structural coherence. Through extensive experiments on shape generation and manipulation tasks, we demonstrate that our approach significantly outperforms previous methods in both quality and controllability of part-level operations. Our code will be made publicly available.

Related papers

• MARS: Mesh AutoRegressive Model for 3D Shape Detailization [85.95365919236212]
  We introduce MARS, a novel approach for 3D shape detailization.<n>We propose a mesh autoregressive model capable of generating such latent representations through next-LOD token prediction.<n>Experiments conducted on the challenging 3D Shape Detailization benchmark demonstrate that our proposed MARS model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-02-17T03:12:16Z)
• JADE: Joint-aware Latent Diffusion for 3D Human Generative Modeling [62.77347895550087]
  We introduce JADE, a generative framework that learns the variations of human shapes with fined-grained control.<n>Our key insight is a joint-aware latent representation that decomposes human bodies into skeleton structures.<n>To generate coherent and plausible human shapes under our proposed decomposition, we also present a cascaded pipeline.
  arXiv  Detail & Related papers  (2024-12-29T14:18:35Z)
• DetailGen3D: Generative 3D Geometry Enhancement via Data-Dependent Flow [44.72037991063735]
  DetailGen3D is a generative approach specifically designed to enhance generated 3D shapes.<n>Our key insight is to model the coarse-to-fine transformation directly through data-dependent flows in latent space.<n>We introduce a token matching strategy that ensures accurate spatial correspondence during refinement.
  arXiv  Detail & Related papers  (2024-11-25T17:08:17Z)
• Topology-Aware Latent Diffusion for 3D Shape Generation [20.358373670117537]
  We introduce a new generative model that combines latent diffusion with persistent homology to create 3D shapes with high diversity. Our method involves representing 3D shapes as implicit fields, then employing persistent homology to extract topological features.
  arXiv  Detail & Related papers  (2024-01-31T05:13:53Z)
• Learning Modulated Transformation in GANs [69.95217723100413]
  We equip the generator in generative adversarial networks (GANs) with a plug-and-play module, termed as modulated transformation module (MTM) MTM predicts spatial offsets under the control of latent codes, based on which the convolution operation can be applied at variable locations. It is noteworthy that towards human generation on the challenging TaiChi dataset, we improve the FID of StyleGAN3 from 21.36 to 13.60, demonstrating the efficacy of learning modulated geometry transformation.
  arXiv  Detail & Related papers  (2023-08-29T17:51:22Z)
• Learning Versatile 3D Shape Generation with Improved AR Models [91.87115744375052]
  Auto-regressive (AR) models have achieved impressive results in 2D image generation by modeling joint distributions in the grid space. We propose the Improved Auto-regressive Model (ImAM) for 3D shape generation, which applies discrete representation learning based on a latent vector instead of volumetric grids.
  arXiv  Detail & Related papers  (2023-03-26T12:03:18Z)
• 3DQD: Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Process [32.3773514247982]
  We develop a generalized 3D shape generation prior model tailored for multiple 3D tasks. Designs jointly equip our proposed 3D shape prior model with high-fidelity, diverse features as well as the capability of cross-modality alignment.
  arXiv  Detail & Related papers  (2023-03-18T12:50:29Z)
• SP-GAN: Sphere-Guided 3D Shape Generation and Manipulation [50.53931728235875]
  We present SP-GAN, a new unsupervised sphere-guided generative model for direct synthesis of 3D shapes in the form of point clouds. Compared with existing models, SP-GAN is able to synthesize diverse and high-quality shapes with fine details.
  arXiv  Detail & Related papers  (2021-08-10T06:49:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.