Scaling Mesh Generation via Compressive Tokenization

• URL: http://arxiv.org/abs/2411.07025v1
• Date: Mon, 11 Nov 2024 14:30:35 GMT
• Title: Scaling Mesh Generation via Compressive Tokenization
• Authors: Haohan Weng, Zibo Zhao, Biwen Lei, Xianghui Yang, Jian Liu, Zeqiang Lai, Zhuo Chen, Yuhong Liu, Jie Jiang, Chunchao Guo, Tong Zhang, Shenghua Gao, C. L. Philip Chen,
• Abstract summary: We propose a compressive yet effective mesh representation, Blocked and Patchified Tokenization (BPT) BPT compresses mesh sequences by employing block-wise indexing and patch aggregation, reducing their length by approximately 75% compared to the original sequences. Empowered with the BPT, we have built a foundation mesh generative model training on scaled mesh data to support flexible control for point clouds and images.
• Score: 66.05639158028343
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a compressive yet effective mesh representation, Blocked and Patchified Tokenization (BPT), facilitating the generation of meshes exceeding 8k faces. BPT compresses mesh sequences by employing block-wise indexing and patch aggregation, reducing their length by approximately 75\% compared to the original sequences. This compression milestone unlocks the potential to utilize mesh data with significantly more faces, thereby enhancing detail richness and improving generation robustness. Empowered with the BPT, we have built a foundation mesh generative model training on scaled mesh data to support flexible control for point clouds and images. Our model demonstrates the capability to generate meshes with intricate details and accurate topology, achieving SoTA performance on mesh generation and reaching the level for direct product usage.

Related papers

• Mesh Silksong: Auto-Regressive Mesh Generation as Weaving Silk [25.158463603708245]
  Mesh Silksong is a compact and efficient mesh representation tailored to generate the polygon mesh in an auto-regressive manner akin to silk weaving.<n>We show that Mesh Silksong produces polygon meshes with superior geometric properties, including manifold topology, watertight detection, and consistent face normals.
  arXiv  Detail & Related papers  (2025-07-03T09:34:24Z)
• FoldNet: Learning Generalizable Closed-Loop Policy for Garment Folding via Keypoint-Driven Asset and Demonstration Synthesis [9.22657317122778]
  We present a synthetic garment dataset that can be used for robotic garment folding.<n>We generate folding demonstrations in simulation and train folding policies via closed-loop imitation learning.<n> KG-DAgger significantly improves the model performance, boosting the real-world success rate by 25%.
  arXiv  Detail & Related papers  (2025-05-14T03:34:30Z)
• MeshCraft: Exploring Efficient and Controllable Mesh Generation with Flow-based DiTs [79.45006864728893]
  MeshCraft is a framework for efficient and controllable mesh generation. It uses continuous spatial diffusion to generate discrete triangle faces. It can generate an 800-face mesh in just 3.2 seconds.
  arXiv  Detail & Related papers  (2025-03-29T09:21:50Z)
• TreeMeshGPT: Artistic Mesh Generation with Autoregressive Tree Sequencing [47.919057306538626]
  TreeMeshGPT is an autoregressive Transformer designed to generate artistic meshes aligned with input point clouds. Our approach represents each triangular face with two tokens, achieving a compression rate of approximately 22%. Our method generates mesh with strong normal orientation constraints, minimizing flipped normals commonly encountered in previous methods.
  arXiv  Detail & Related papers  (2025-03-14T17:48:06Z)
• Comb Tensor Networks vs. Matrix Product States: Enhanced Efficiency in High-Dimensional Spaces [0.0]
  We show that a comb-shaped tensor network architecture can yield more efficient contractions than a standard MPS. This finding suggests that for continuous and high-dimensional data distributions, transitioning from MPS to a comb tensor network representation can substantially reduce computational overhead while maintaining accuracy.
  arXiv  Detail & Related papers  (2024-12-08T20:28:49Z)
• SpaceMesh: A Continuous Representation for Learning Manifold Surface Meshes [61.110517195874074]
  We present a scheme to directly generate manifold, polygonal meshes of complex connectivity as the output of a neural network. Our key innovation is to define a continuous latent connectivity space at each mesh, which implies the discrete mesh. In applications, this approach not only yields high-quality outputs from generative models, but also enables directly learning challenging geometry processing tasks such as mesh repair.
  arXiv  Detail & Related papers  (2024-09-30T17:59:03Z)
• MeshAnything V2: Artist-Created Mesh Generation With Adjacent Mesh Tokenization [65.15226276553891]
  MeshAnything V2 is an advanced mesh generation model designed to create Artist-Created Meshes. A key innovation behind MeshAnything V2 is our novel Adjacent Mesh Tokenization (AMT) method.
  arXiv  Detail & Related papers  (2024-08-05T15:33:45Z)
• PivotMesh: Generic 3D Mesh Generation via Pivot Vertices Guidance [66.40153183581894]
  We introduce a generic and scalable mesh generation framework PivotMesh. PivotMesh makes an initial attempt to extend the native mesh generation to large-scale datasets. We show that PivotMesh can generate compact and sharp 3D meshes across various categories.
  arXiv  Detail & Related papers  (2024-05-27T07:13:13Z)
• MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers [32.169007676811404]
  MeshGPT is a new approach for generating triangle meshes that reflects the compactness typical of artist-created meshes. Inspired by recent advances in powerful large language models, we adopt a sequence-based approach to autoregressively generate triangle meshes as triangles.
  arXiv  Detail & Related papers  (2023-11-27T01:20:11Z)
• Controllable Mesh Generation Through Sparse Latent Point Diffusion Models [105.83595545314334]
  We design a novel sparse latent point diffusion model for mesh generation. Our key insight is to regard point clouds as an intermediate representation of meshes, and model the distribution of point clouds instead. Our proposed sparse latent point diffusion model achieves superior performance in terms of generation quality and controllability.
  arXiv  Detail & Related papers  (2023-03-14T14:25:29Z)
• Mesh Draping: Parametrization-Free Neural Mesh Transfer [92.55503085245304]
  Mesh Draping is a neural method for transferring existing mesh structure from one shape to another. We show that by leveraging gradually increasing frequencies to guide the neural optimization, we are able to achieve stable and high quality mesh transfer.
  arXiv  Detail & Related papers  (2021-10-11T17:24:52Z)
• Deep Deformation Detail Synthesis for Thin Shell Models [47.442883859643004]
  In physics-based cloth animation, rich folds and detailed wrinkles are achieved at the cost of expensive computational resources and huge labor tuning. We develop a temporally and spatially as-consistent-as-possible deformation representation (named TS-ACAP) and a DeformTransformer network to learn the mapping from low-resolution meshes to detailed ones. Our method is able to produce reliable and realistic animations in various datasets at high frame rates: 10 35 times faster than physics-based simulation, with superior detail synthesis abilities than existing methods.
  arXiv  Detail & Related papers  (2021-02-23T08:09:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.