MeshCraft: Exploring Efficient and Controllable Mesh Generation with Flow-based DiTs

• URL: http://arxiv.org/abs/2503.23022v1
• Date: Sat, 29 Mar 2025 09:21:50 GMT
• Title: MeshCraft: Exploring Efficient and Controllable Mesh Generation with Flow-based DiTs
• Authors: Xianglong He, Junyi Chen, Di Huang, Zexiang Liu, Xiaoshui Huang, Wanli Ouyang, Chun Yuan, Yangguang Li,
• Abstract summary: MeshCraft is a framework for efficient and controllable mesh generation.<n>It uses continuous spatial diffusion to generate discrete triangle faces.<n>It can generate an 800-face mesh in just 3.2 seconds.
• Score: 79.45006864728893
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the domain of 3D content creation, achieving optimal mesh topology through AI models has long been a pursuit for 3D artists. Previous methods, such as MeshGPT, have explored the generation of ready-to-use 3D objects via mesh auto-regressive techniques. While these methods produce visually impressive results, their reliance on token-by-token predictions in the auto-regressive process leads to several significant limitations. These include extremely slow generation speeds and an uncontrollable number of mesh faces. In this paper, we introduce MeshCraft, a novel framework for efficient and controllable mesh generation, which leverages continuous spatial diffusion to generate discrete triangle faces. Specifically, MeshCraft consists of two core components: 1) a transformer-based VAE that encodes raw meshes into continuous face-level tokens and decodes them back to the original meshes, and 2) a flow-based diffusion transformer conditioned on the number of faces, enabling the generation of high-quality 3D meshes with a predefined number of faces. By utilizing the diffusion model for the simultaneous generation of the entire mesh topology, MeshCraft achieves high-fidelity mesh generation at significantly faster speeds compared to auto-regressive methods. Specifically, MeshCraft can generate an 800-face mesh in just 3.2 seconds (35$\times$ faster than existing baselines). Extensive experiments demonstrate that MeshCraft outperforms state-of-the-art techniques in both qualitative and quantitative evaluations on ShapeNet dataset and demonstrates superior performance on Objaverse dataset. Moreover, it integrates seamlessly with existing conditional guidance strategies, showcasing its potential to relieve artists from the time-consuming manual work involved in mesh creation.

Related papers

• DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learning [21.77406648840365]
  DeepMesh is a framework that optimize mesh generation through two key innovations. It incorporates a novel tokenization algorithm, along with improvements in data curation and processing. It generates meshes with intricate details and precise topology, outperforming state-of-the-art methods in both precision and quality.
  arXiv  Detail & Related papers  (2025-03-19T14:39:30Z)
• MeshPad: Interactive Sketch-Conditioned Artist-Designed Mesh Generation and Editing [64.84885028248395]
  MeshPad is a generative approach that creates 3D meshes from sketch inputs. We focus on enabling consistent edits by decomposing editing into 'deletion' of regions of a mesh, followed by 'addition' of new mesh geometry. Our approach is based on a triangle sequence-based mesh representation, exploiting a large Transformer model for mesh triangle addition and deletion.
  arXiv  Detail & Related papers  (2025-03-03T11:27:44Z)
• GenUDC: High Quality 3D Mesh Generation with Unsigned Dual Contouring Representation [13.923644541595893]
  3D generative models generate high-quality meshes with complex structures and realistic surfaces. We propose the GenUDC framework to address these challenges by leveraging the Unsigned Dual Contouring (UDC) as the mesh representation. In addition, GenUDC adopts a two-stage, coarse-to-fine generative process for 3D mesh generation.
  arXiv  Detail & Related papers  (2024-10-23T11:59:49Z)
• MeshAnything V2: Artist-Created Mesh Generation With Adjacent Mesh Tokenization [65.15226276553891]
  MeshAnything V2 is an advanced mesh generation model designed to create Artist-Created Meshes.<n>A key innovation behind MeshAnything V2 is our novel Adjacent Mesh Tokenization (AMT) method.
  arXiv  Detail & Related papers  (2024-08-05T15:33:45Z)
• MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers [76.70891862458384]
  We introduce MeshAnything, a model that treats mesh extraction as a generation problem. By converting 3D assets in any 3D representation into AMs, MeshAnything can be integrated with various 3D asset production methods. Our method generates AMs with hundreds of times fewer faces, significantly improving storage, rendering, and simulation efficiencies.
  arXiv  Detail & Related papers  (2024-06-14T16:30:25Z)
• MeshXL: Neural Coordinate Field for Generative 3D Foundation Models [51.1972329762843]
  We present a family of generative pre-trained auto-regressive models, which addresses the process of 3D mesh generation with modern large language model approaches. MeshXL is able to generate high-quality 3D meshes, and can also serve as foundation models for various down-stream applications.
  arXiv  Detail & Related papers  (2024-05-31T14:35:35Z)
• PivotMesh: Generic 3D Mesh Generation via Pivot Vertices Guidance [66.40153183581894]
  We introduce a generic and scalable mesh generation framework PivotMesh. PivotMesh makes an initial attempt to extend the native mesh generation to large-scale datasets. We show that PivotMesh can generate compact and sharp 3D meshes across various categories.
  arXiv  Detail & Related papers  (2024-05-27T07:13:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.