MARS: Mesh AutoRegressive Model for 3D Shape Detailization

• URL: http://arxiv.org/abs/2502.11390v1
• Date: Mon, 17 Feb 2025 03:12:16 GMT
• Title: MARS: Mesh AutoRegressive Model for 3D Shape Detailization
• Authors: Jingnan Gao, Weizhe Liu, Weixuan Sun, Senbo Wang, Xibin Song, Taizhang Shang, Shenzhou Chen, Hongdong Li, Xiaokang Yang, Yichao Yan, Pan Ji,
• Abstract summary: We introduce MARS, a novel approach for 3D shape detailization.<n>We propose a mesh autoregressive model capable of generating such latent representations through next-LOD token prediction.<n>Experiments conducted on the challenging 3D Shape Detailization benchmark demonstrate that our proposed MARS model achieves state-of-the-art performance.
• Score: 85.95365919236212
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: State-of-the-art methods for mesh detailization predominantly utilize Generative Adversarial Networks (GANs) to generate detailed meshes from coarse ones. These methods typically learn a specific style code for each category or similar categories without enforcing geometry supervision across different Levels of Detail (LODs). Consequently, such methods often fail to generalize across a broader range of categories and cannot ensure shape consistency throughout the detailization process. In this paper, we introduce MARS, a novel approach for 3D shape detailization. Our method capitalizes on a novel multi-LOD, multi-category mesh representation to learn shape-consistent mesh representations in latent space across different LODs. We further propose a mesh autoregressive model capable of generating such latent representations through next-LOD token prediction. This approach significantly enhances the realism of the generated shapes. Extensive experiments conducted on the challenging 3D Shape Detailization benchmark demonstrate that our proposed MARS model achieves state-of-the-art performance, surpassing existing methods in both qualitative and quantitative assessments. Notably, the model's capability to generate fine-grained details while preserving the overall shape integrity is particularly commendable.

Related papers

• PRISM: Probabilistic Representation for Integrated Shape Modeling and Generation [79.46526296655776]
  PRISM is a novel approach for 3D shape generation that integrates categorical diffusion models with Statistical Shape Models (SSM) and Gaussian Mixture Models (GMM) Our method employs compositional SSMs to capture part-level geometric variations and uses GMM to represent part semantics in a continuous space. Our approach significantly outperforms previous methods in both quality and controllability of part-level operations.
  arXiv  Detail & Related papers  (2025-04-06T11:48:08Z)
• DetailGen3D: Generative 3D Geometry Enhancement via Data-Dependent Flow [44.72037991063735]
  DetailGen3D is a generative approach specifically designed to enhance generated 3D shapes.<n>Our key insight is to model the coarse-to-fine transformation directly through data-dependent flows in latent space.<n>We introduce a token matching strategy that ensures accurate spatial correspondence during refinement.
  arXiv  Detail & Related papers  (2024-11-25T17:08:17Z)
• WeatherDG: LLM-assisted Diffusion Model for Procedural Weather Generation in Domain-Generalized Semantic Segmentation [4.141230571282547]
  We propose a novel approach, namely WeatherDG, that can generate realistic, weather-diverse, and driving-screen images.<n>We first fine-tune the SD with source data, aligning the content and layout of generated samples with real-world driving scenarios.<n>We introduce a balanced generation strategy, which encourages the SD to generate high-quality objects of tailed classes under various weather conditions.
  arXiv  Detail & Related papers  (2024-10-15T21:29:26Z)
• MVGaussian: High-Fidelity text-to-3D Content Generation with Multi-View Guidance and Surface Densification [13.872254142378772]
  This paper introduces a unified framework for text-to-3D content generation. Our approach utilizes multi-view guidance to iteratively form the structure of the 3D model. We also introduce a novel densification algorithm that aligns gaussians close to the surface.
  arXiv  Detail & Related papers  (2024-09-10T16:16:34Z)
• A Latent Implicit 3D Shape Model for Multiple Levels of Detail [95.56814217356667]
  Implicit neural representations map a shape-specific latent code and a 3D coordinate to its corresponding signed distance (SDF) value. This approach only offers a single level of detail. We propose a new shape modeling approach, which enables multiple levels of detail and guarantees a smooth surface at each level.
  arXiv  Detail & Related papers  (2024-09-10T05:57:58Z)
• DECOLLAGE: 3D Detailization by Controllable, Localized, and Learned Geometry Enhancement [38.719572669042925]
  We present a 3D modeling method which enables end-users to refine or detailize 3D shapes using machine learning. We show that our ability to localize details enables novel interactive creative and applications.
  arXiv  Detail & Related papers  (2024-09-10T00:51:49Z)
• Surf-D: Generating High-Quality Surfaces of Arbitrary Topologies Using Diffusion Models [83.35835521670955]
  Surf-D is a novel method for generating high-quality 3D shapes as Surfaces with arbitrary topologies. We use the Unsigned Distance Field (UDF) as our surface representation to accommodate arbitrary topologies. We also propose a new pipeline that employs a point-based AutoEncoder to learn a compact and continuous latent space for accurately encoding UDF.
  arXiv  Detail & Related papers  (2023-11-28T18:56:01Z)
• T1: Scaling Diffusion Probabilistic Fields to High-Resolution on Unified Visual Modalities [69.16656086708291]
  Diffusion Probabilistic Field (DPF) models the distribution of continuous functions defined over metric spaces. We propose a new model comprising of a view-wise sampling algorithm to focus on local structure learning. The model can be scaled to generate high-resolution data while unifying multiple modalities.
  arXiv  Detail & Related papers  (2023-05-24T03:32:03Z)
• Learning Versatile 3D Shape Generation with Improved AR Models [91.87115744375052]
  Auto-regressive (AR) models have achieved impressive results in 2D image generation by modeling joint distributions in the grid space. We propose the Improved Auto-regressive Model (ImAM) for 3D shape generation, which applies discrete representation learning based on a latent vector instead of volumetric grids.
  arXiv  Detail & Related papers  (2023-03-26T12:03:18Z)
• Manifold Topology Divergence: a Framework for Comparing Data Manifolds [109.0784952256104]
  We develop a framework for comparing data manifold, aimed at the evaluation of deep generative models. Based on the Cross-Barcode, we introduce the Manifold Topology Divergence score (MTop-Divergence) We demonstrate that the MTop-Divergence accurately detects various degrees of mode-dropping, intra-mode collapse, mode invention, and image disturbance.
  arXiv  Detail & Related papers  (2021-06-08T00:30:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.