Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabrication

• URL: http://arxiv.org/abs/2405.18515v1
• Date: Tue, 28 May 2024 18:33:18 GMT
• Title: Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabrication
• Authors: Yunuo Chen, Tianyi Xie, Zeshun Zong, Xuan Li, Feng Gao, Yin Yang, Ying Nian Wu, Chenfanfu Jiang,
• Abstract summary: We introduce Atlas3D, an automatic and easy-to-implement text-to-3D method. Our approach combines a novel differentiable simulation-based loss function with physically inspired regularization. We verify Atlas3D's efficacy through extensive generation tasks and validate the resulting 3D models in both simulated and real-world environments.
• Score: 50.541882834405946
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing diffusion-based text-to-3D generation methods primarily focus on producing visually realistic shapes and appearances, often neglecting the physical constraints necessary for downstream tasks. Generated models frequently fail to maintain balance when placed in physics-based simulations or 3D printed. This balance is crucial for satisfying user design intentions in interactive gaming, embodied AI, and robotics, where stable models are needed for reliable interaction. Additionally, stable models ensure that 3D-printed objects, such as figurines for home decoration, can stand on their own without requiring additional supports. To fill this gap, we introduce Atlas3D, an automatic and easy-to-implement method that enhances existing Score Distillation Sampling (SDS)-based text-to-3D tools. Atlas3D ensures the generation of self-supporting 3D models that adhere to physical laws of stability under gravity, contact, and friction. Our approach combines a novel differentiable simulation-based loss function with physically inspired regularization, serving as either a refinement or a post-processing module for existing frameworks. We verify Atlas3D's efficacy through extensive generation tasks and validate the resulting 3D models in both simulated and real-world environments.

Related papers

• SYM3D: Learning Symmetric Triplanes for Better 3D-Awareness of GANs [5.84660008137615]
  We propose SYM3D, a novel 3D-aware GAN designed to leverage the prevalental symmetry structure found in natural and man-made objects. We evaluate SYM3D on both synthetic (ShapeNet Chairs, Cars, and Airplanes) and real-world datasets.
  arXiv  Detail & Related papers  (2024-06-10T16:24:07Z)
• Pushing Auto-regressive Models for 3D Shape Generation at Capacity and Scalability [118.26563926533517]
  Auto-regressive models have achieved impressive results in 2D image generation by modeling joint distributions in grid space. We extend auto-regressive models to 3D domains, and seek a stronger ability of 3D shape generation by improving auto-regressive models at capacity and scalability simultaneously.
  arXiv  Detail & Related papers  (2024-02-19T15:33:09Z)
• En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Data [36.51674664590734]
  We present En3D, an enhanced izable scheme for high-qualityd 3D human avatars. Unlike previous works that rely on scarce 3D datasets or limited 2D collections with imbalance viewing angles and pose priors, our approach aims to develop a zero-shot 3D capable of producing 3D humans.
  arXiv  Detail & Related papers  (2024-01-02T12:06:31Z)
• FILP-3D: Enhancing 3D Few-shot Class-incremental Learning with Pre-trained Vision-Language Models [62.663113296987085]
  Few-shot class-incremental learning aims to mitigate the catastrophic forgetting issue when a model is incrementally trained on limited data. We introduce two novel components: the Redundant Feature Eliminator (RFE) and the Spatial Noise Compensator (SNC) Considering the imbalance in existing 3D datasets, we also propose new evaluation metrics that offer a more nuanced assessment of a 3D FSCIL model.
  arXiv  Detail & Related papers  (2023-12-28T14:52:07Z)
• LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching [33.696757740830506]
  Recent advancements in text-to-3D generation have shown promise. Many methods base themselves on Score Distillation Sampling (SDS) We propose Interval Score Matching (ISM) to counteract over-smoothing.
  arXiv  Detail & Related papers  (2023-11-19T09:59:09Z)
• AutoDecoding Latent 3D Diffusion Models [95.7279510847827]
  We present a novel approach to the generation of static and articulated 3D assets that has a 3D autodecoder at its core. The 3D autodecoder framework embeds properties learned from the target dataset in the latent space. We then identify the appropriate intermediate volumetric latent space, and introduce robust normalization and de-normalization operations.
  arXiv  Detail & Related papers  (2023-07-07T17:59:14Z)
• MoDA: Modeling Deformable 3D Objects from Casual Videos [84.29654142118018]
  We propose neural dual quaternion blend skinning (NeuDBS) to achieve 3D point deformation without skin-collapsing artifacts. In the endeavor to register 2D pixels across different frames, we establish a correspondence between canonical feature embeddings that encodes 3D points within the canonical space. Our approach can reconstruct 3D models for humans and animals with better qualitative and quantitative performance than state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-04-17T13:49:04Z)
• GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images [72.15855070133425]
  We introduce GET3D, a Generative model that directly generates Explicit Textured 3D meshes with complex topology, rich geometric details, and high-fidelity textures. GET3D is able to generate high-quality 3D textured meshes, ranging from cars, chairs, animals, motorbikes and human characters to buildings.
  arXiv  Detail & Related papers  (2022-09-22T17:16:19Z)
• I-nteract 2.0: A Cyber-Physical System to Design 3D Models using Mixed Reality Technologies and Deep Learning for Additive Manufacturing [2.7986973063309875]
  I-nteract is a cyber-physical system that enables real-time interaction with both virtual and real artifacts to design 3D models for additive manufacturing. This paper presents novel advances in the development of the interaction platform I-nteract to generate 3D models using both constructive solid geometry and artificial intelligence.
  arXiv  Detail & Related papers  (2020-10-21T14:13:21Z)
• Amodal 3D Reconstruction for Robotic Manipulation via Stability and Connectivity [3.359622001455893]
  Learning-based 3D object reconstruction enables single- or few-shot estimation of 3D object models. Existing 3D reconstruction techniques optimize for visual reconstruction fidelity, typically measured by chamfer distance or voxel IOU. We propose ARM, an amodal 3D reconstruction system that introduces a stability prior over object shapes, (2) a connectivity prior, and (3) a multi-channel input representation.
  arXiv  Detail & Related papers  (2020-09-28T08:52:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.