ArtFormer: Controllable Generation of Diverse 3D Articulated Objects

• URL: http://arxiv.org/abs/2412.07237v3
• Date: Thu, 03 Apr 2025 14:16:29 GMT
• Title: ArtFormer: Controllable Generation of Diverse 3D Articulated Objects
• Authors: Jiayi Su, Youhe Feng, Zheng Li, Jinhua Song, Yangfan He, Botao Ren, Botian Xu,
• Abstract summary: This paper presents a novel framework for modeling and conditional generation of 3D articulated objects.<n>We parameterize an articulated object as a tree of tokens and employ a transformer to generate both the object's high-level geometry code and its kinematic relations.<n>Our approach enables the generation of diverse objects with high-quality geometry and varying number of parts.
• Score: 5.320860732053524
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This paper presents a novel framework for modeling and conditional generation of 3D articulated objects. Troubled by flexibility-quality tradeoffs, existing methods are often limited to using predefined structures or retrieving shapes from static datasets. To address these challenges, we parameterize an articulated object as a tree of tokens and employ a transformer to generate both the object's high-level geometry code and its kinematic relations. Subsequently, each sub-part's geometry is further decoded using a signed-distance-function (SDF) shape prior, facilitating the synthesis of high-quality 3D shapes. Our approach enables the generation of diverse objects with high-quality geometry and varying number of parts. Comprehensive experiments on conditional generation from text descriptions demonstrate the effectiveness and flexibility of our method.

Related papers

• HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation [50.206100327643284]
  HiScene is a novel hierarchical framework that bridges the gap between 2D image generation and 3D object generation. We generate 3D content that aligns with 2D representations while maintaining compositional structure.
  arXiv  Detail & Related papers  (2025-04-17T16:33:39Z)
• A3D: Does Diffusion Dream about 3D Alignment? [73.97853402817405]
  We tackle the problem of text-driven 3D generation from a geometry alignment perspective. Given a set of text prompts, we aim to generate a collection of objects with semantically corresponding parts aligned across them. We propose to embed these objects into a common latent space and optimize the continuous transitions between these objects.
  arXiv  Detail & Related papers  (2024-06-21T09:49:34Z)
• NeuSDFusion: A Spatial-Aware Generative Model for 3D Shape Completion, Reconstruction, and Generation [52.772319840580074]
  3D shape generation aims to produce innovative 3D content adhering to specific conditions and constraints. Existing methods often decompose 3D shapes into a sequence of localized components, treating each element in isolation. We introduce a novel spatial-aware 3D shape generation framework that leverages 2D plane representations for enhanced 3D shape modeling.
  arXiv  Detail & Related papers  (2024-03-27T04:09:34Z)
• CG3D: Compositional Generation for Text-to-3D via Gaussian Splatting [57.14748263512924]
  CG3D is a method for compositionally generating scalable 3D assets. Gamma radiance fields, parameterized to allow for compositions of objects, possess the capability to enable semantically and physically consistent scenes.
  arXiv  Detail & Related papers  (2023-11-29T18:55:38Z)
• DeFormer: Integrating Transformers with Deformable Models for 3D Shape Abstraction from a Single Image [31.154786931081087]
  We propose a novel bi-channel Transformer architecture, integrated with parameterized deformable models, to simultaneously estimate the global and local deformations of primitives. DeFormer achieves better reconstruction accuracy over the state-of-the-art, and visualizes with consistent semantic correspondences for improved interpretability.
  arXiv  Detail & Related papers  (2023-09-22T02:46:43Z)
• RISA-Net: Rotation-Invariant Structure-Aware Network for Fine-Grained 3D Shape Retrieval [46.02391761751015]
  Fine-grained 3D shape retrieval aims to retrieve 3D shapes similar to a query shape in a repository with models belonging to the same class. We introduce a novel deep architecture, RISA-Net, which learns rotation invariant 3D shape descriptors. Our method is able to learn the importance of geometric and structural information of all the parts when generating the final compact latent feature of a 3D shape.
  arXiv  Detail & Related papers  (2020-10-02T13:06:12Z)
• Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images [64.53227129573293]
  We investigate the problem of learning to generate 3D parametric surface representations for novel object instances, as seen from one or more views. We design neural networks capable of generating high-quality parametric 3D surfaces which are consistent between views. Our method is supervised and trained on a public dataset of shapes from common object categories.
  arXiv  Detail & Related papers  (2020-08-18T06:33:40Z)
• DSG-Net: Learning Disentangled Structure and Geometry for 3D Shape Generation [98.96086261213578]
  We introduce DSG-Net, a deep neural network that learns a disentangled structured and geometric mesh representation for 3D shapes. This supports a range of novel shape generation applications with disentangled control, such as of structure (geometry) while keeping geometry (structure) unchanged. Our method not only supports controllable generation applications but also produces high-quality synthesized shapes, outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-12T17:06:51Z)
• Learning Unsupervised Hierarchical Part Decomposition of 3D Objects from a Single RGB Image [102.44347847154867]
  We propose a novel formulation that allows to jointly recover the geometry of a 3D object as a set of primitives. Our model recovers the higher level structural decomposition of various objects in the form of a binary tree of primitives. Our experiments on the ShapeNet and D-FAUST datasets demonstrate that considering the organization of parts indeed facilitates reasoning about 3D geometry.
  arXiv  Detail & Related papers  (2020-04-02T17:58:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.