ThemeStation: Generating Theme-Aware 3D Assets from Few Exemplars

• URL: http://arxiv.org/abs/2403.15383v2
• Date: Wed, 15 May 2024 06:56:30 GMT
• Title: ThemeStation: Generating Theme-Aware 3D Assets from Few Exemplars
• Authors: Zhenwei Wang, Tengfei Wang, Gerhard Hancke, Ziwei Liu, Rynson W. H. Lau,
• Abstract summary: Real-world applications often require a large gallery of 3D assets that share a consistent theme. We present ThemeStation, a novel approach for theme-aware 3D-to-3D generation.
• Score: 62.34862776670368
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world applications often require a large gallery of 3D assets that share a consistent theme. While remarkable advances have been made in general 3D content creation from text or image, synthesizing customized 3D assets following the shared theme of input 3D exemplars remains an open and challenging problem. In this work, we present ThemeStation, a novel approach for theme-aware 3D-to-3D generation. ThemeStation synthesizes customized 3D assets based on given few exemplars with two goals: 1) unity for generating 3D assets that thematically align with the given exemplars and 2) diversity for generating 3D assets with a high degree of variations. To this end, we design a two-stage framework that draws a concept image first, followed by a reference-informed 3D modeling stage. We propose a novel dual score distillation (DSD) loss to jointly leverage priors from both the input exemplars and the synthesized concept image. Extensive experiments and user studies confirm that ThemeStation surpasses prior works in producing diverse theme-aware 3D models with impressive quality. ThemeStation also enables various applications such as controllable 3D-to-3D generation.

Related papers

• Constructing a 3D Town from a Single Image [23.231661811526955]
  3DTown is a training-free framework designed to synthesize realistic and coherent 3D scenes from a single top-down view.<n>We decompose the input image into overlapping regions and generate each using a pretrained 3D object generator.<n>Our results demonstrate that high-quality 3D town generation is achievable from a single image using a principled, training-free approach.
  arXiv  Detail & Related papers  (2025-05-21T17:10:47Z)
• TAR3D: Creating High-Quality 3D Assets via Next-Part Prediction [137.34863114016483]
  TAR3D is a novel framework that consists of a 3D-aware Vector Quantized-Variational AutoEncoder (VQ-VAE) and a Generative Pre-trained Transformer (GPT) We show that TAR3D can achieve superior generation quality over existing methods in text-to-3D and image-to-3D tasks.
  arXiv  Detail & Related papers  (2024-12-22T08:28:20Z)
• MUSES: 3D-Controllable Image Generation via Multi-Modal Agent Collaboration [29.657854912416038]
  We introduce a generic AI system, namely MUSES, for 3D-controllable image generation from user queries. By mimicking the collaboration of human professionals, this multi-modal agent pipeline facilitates the effective and automatic creation of images with 3D-controllable objects. We construct a new benchmark of T2I-3DisBench (3D image scene), which describes diverse 3D image scenes with 50 detailed prompts.
  arXiv  Detail & Related papers  (2024-08-20T07:37:23Z)
• Meta 3D Gen [57.313835190702484]
  3DGen offers 3D asset creation with high prompt fidelity and high-quality 3D shapes and textures in under a minute. 3DGen supports physically-based rendering (PBR), necessary for 3D asset relighting in real-world applications.
  arXiv  Detail & Related papers  (2024-07-02T18:37:52Z)
• ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware Diffusion Guidance [76.7746870349809]
  We present ComboVerse, a 3D generation framework that produces high-quality 3D assets with complex compositions by learning to combine multiple models. Our proposed framework emphasizes spatial alignment of objects, compared with standard score distillation sampling.
  arXiv  Detail & Related papers  (2024-03-19T03:39:43Z)
• Make-Your-3D: Fast and Consistent Subject-Driven 3D Content Generation [12.693847842218604]
  We introduce a novel 3D customization method, dubbed Make-Your-3D, that can personalize high-fidelity and consistent 3D content within 5 minutes. Our key insight is to harmonize the distributions of a multi-view diffusion model and an identity-specific 2D generative model, aligning them with the distribution of the desired 3D subject. Our method can produce high-quality, consistent, and subject-specific 3D content with text-driven modifications that are unseen in subject image.
  arXiv  Detail & Related papers  (2024-03-14T17:57:04Z)
• One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion [32.29687304798145]
  One-2-3-45++ is an innovative method that transforms a single image into a detailed 3D textured mesh in approximately one minute. Our approach aims to fully harness the extensive knowledge embedded in 2D diffusion models and priors from valuable yet limited 3D data.
  arXiv  Detail & Related papers  (2023-11-14T03:40:25Z)
• Large-Vocabulary 3D Diffusion Model with Transformer [57.076986347047]
  We introduce a diffusion-based feed-forward framework for synthesizing massive categories of real-world 3D objects with a single generative model. We propose a novel triplane-based 3D-aware Diffusion model with TransFormer, DiffTF, for handling challenges via three aspects. Experiments on ShapeNet and OmniObject3D convincingly demonstrate that a single DiffTF model achieves state-of-the-art large-vocabulary 3D object generation performance.
  arXiv  Detail & Related papers  (2023-09-14T17:59:53Z)
• CC3D: Layout-Conditioned Generation of Compositional 3D Scenes [49.281006972028194]
  We introduce CC3D, a conditional generative model that synthesizes complex 3D scenes conditioned on 2D semantic scene layouts. Our evaluations on synthetic 3D-FRONT and real-world KITTI-360 datasets demonstrate that our model generates scenes of improved visual and geometric quality.
  arXiv  Detail & Related papers  (2023-03-21T17:59:02Z)
• GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images [72.15855070133425]
  We introduce GET3D, a Generative model that directly generates Explicit Textured 3D meshes with complex topology, rich geometric details, and high-fidelity textures. GET3D is able to generate high-quality 3D textured meshes, ranging from cars, chairs, animals, motorbikes and human characters to buildings.
  arXiv  Detail & Related papers  (2022-09-22T17:16:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.