C3DAG: Controlled 3D Animal Generation using 3D pose guidance

• URL: http://arxiv.org/abs/2406.07742v1
• Date: Tue, 11 Jun 2024 21:57:47 GMT
• Title: C3DAG: Controlled 3D Animal Generation using 3D pose guidance
• Abstract summary: C3DAG is a novel pose-Controlled text-to-3D Generation framework. It generates a high quality 3D animal consistent with a given pose.
• Score: 32.067205350291964
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advancements in text-to-3D generation have demonstrated the ability to generate high quality 3D assets. However while generating animals these methods underperform, often portraying inaccurate anatomy and geometry. Towards ameliorating this defect, we present C3DAG, a novel pose-Controlled text-to-3D Animal Generation framework which generates a high quality 3D animal consistent with a given pose. We also introduce an automatic 3D shape creator tool, that allows dynamic pose generation and modification via a web-based tool, and that generates a 3D balloon animal using simple geometries. A NeRF is then initialized using this 3D shape using depth-controlled SDS. In the next stage, the pre-trained NeRF is fine-tuned using quadruped-pose-controlled SDS. The pipeline that we have developed not only produces geometrically and anatomically consistent results, but also renders highly controlled 3D animals, unlike prior methods which do not allow fine-grained pose control.

Related papers

• 3D-ReGen: A Unified 3D Geometry Regeneration Framework [101.26715424980996]
  We introduce 3D-ReGen, a 3D regenerator conditioned on an initial 3D shape.<n>3D-ReGen supports numerous useful tasks, including 3D enhancement, reconstruction, and editing.<n>We evaluate both the geometric consistency and fine-grained quality of 3D-ReGen, achieving state-of-the-art performance in controllable 3D generation.
  arXiv  Detail & Related papers  (2026-04-30T17:18:05Z)
• Generating Surface for Text-to-3D using 2D Gaussian Splatting [7.610379621632961]
  We propose a novel method named DirectGaussian, which focuses on generating the surfaces of 3D objects represented by surfels.<n>In DirectGaussian, we utilize conditional text generation models and the surface of a 3D object is rendered by 2D Gaussian splatting.<n>Our framework is capable of achieving diverse and high-fidelity 3D content creation.
  arXiv  Detail & Related papers  (2025-10-08T12:54:57Z)
• Generating Editable Head Avatars with 3D Gaussian GANs [57.51487984425395]
  Traditional 3D-aware generative adversarial networks (GANs) achieve photorealistic and view-consistent 3D head synthesis. We propose a novel approach that enhances the editability and animation control of 3D head avatars by incorporating 3D Gaussian Splatting (3DGS) as an explicit 3D representation. Our approach delivers high-quality 3D-aware synthesis with state-of-the-art controllability.
  arXiv  Detail & Related papers  (2024-12-26T10:10:03Z)
• TAR3D: Creating High-Quality 3D Assets via Next-Part Prediction [137.34863114016483]
  TAR3D is a novel framework that consists of a 3D-aware Vector Quantized-Variational AutoEncoder (VQ-VAE) and a Generative Pre-trained Transformer (GPT) We show that TAR3D can achieve superior generation quality over existing methods in text-to-3D and image-to-3D tasks.
  arXiv  Detail & Related papers  (2024-12-22T08:28:20Z)
• Deep Geometric Moments Promote Shape Consistency in Text-to-3D Generation [27.43973967994717]
  MT3D is a text-to-3D generative model that leverages a high-fidelity 3D object to overcome viewpoint bias. We employ depth maps derived from a high-quality 3D model as control signals to guarantee that the generated 2D images preserve the fundamental shape and structure. By incorporating geometric details from a 3D asset, MT3D enables the creation of diverse and geometrically consistent objects.
  arXiv  Detail & Related papers  (2024-08-12T06:25:44Z)
• ScalingGaussian: Enhancing 3D Content Creation with Generative Gaussian Splatting [30.99112626706754]
  The creation of high-quality 3D assets is paramount for applications in digital heritage, entertainment, and robotics. Traditionally, this process necessitates skilled professionals and specialized software for modeling. We introduce a novel 3D content creation framework, which generates 3D textures efficiently.
  arXiv  Detail & Related papers  (2024-07-26T18:26:01Z)
• DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D Data [50.164670363633704]
  We present DIRECT-3D, a diffusion-based 3D generative model for creating high-quality 3D assets from text prompts. Our model is directly trained on extensive noisy and unaligned in-the-wild' 3D assets. We achieve state-of-the-art performance in both single-class generation and text-to-3D generation.
  arXiv  Detail & Related papers  (2024-06-06T17:58:15Z)
• Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior [57.986512832738704]
  We present a new framework Sculpt3D that equips the current pipeline with explicit injection of 3D priors from retrieved reference objects without re-training the 2D diffusion model. Specifically, we demonstrate that high-quality and diverse 3D geometry can be guaranteed by keypoints supervision through a sparse ray sampling approach. These two decoupled designs effectively harness 3D information from reference objects to generate 3D objects while preserving the generation quality of the 2D diffusion model.
  arXiv  Detail & Related papers  (2024-03-14T07:39:59Z)
• En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Data [36.51674664590734]
  We present En3D, an enhanced izable scheme for high-qualityd 3D human avatars. Unlike previous works that rely on scarce 3D datasets or limited 2D collections with imbalance viewing angles and pose priors, our approach aims to develop a zero-shot 3D capable of producing 3D humans.
  arXiv  Detail & Related papers  (2024-01-02T12:06:31Z)
• GET3D--: Learning GET3D from Unconstrained Image Collections [27.470617383305726]
  We propose GET3D--, the first method that directly generates textured 3D shapes from 2D images with unknown pose and scale. GET3D-- comprises a 3D shape generator and a learnable camera sampler that captures the 6D external changes on the camera.
  arXiv  Detail & Related papers  (2023-07-27T15:00:54Z)
• Articulated 3D Head Avatar Generation using Text-to-Image Diffusion Models [107.84324544272481]
  The ability to generate diverse 3D articulated head avatars is vital to a plethora of applications, including augmented reality, cinematography, and education. Recent work on text-guided 3D object generation has shown great promise in addressing these needs. We show that our diffusion-based articulated head avatars outperform state-of-the-art approaches for this task.
  arXiv  Detail & Related papers  (2023-07-10T19:15:32Z)
• CGOF++: Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields [52.14985242487535]
  We propose a new conditional 3D face synthesis framework, which enables 3D controllability over generated face images. At its core is a conditional Generative Occupancy Field (cGOF++) that effectively enforces the shape of the generated face to conform to a given 3D Morphable Model (3DMM) mesh. Experiments validate the effectiveness of the proposed method and show more precise 3D controllability than state-of-the-art 2D-based controllable face synthesis methods.
  arXiv  Detail & Related papers  (2022-11-23T19:02:50Z)
• XDGAN: Multi-Modal 3D Shape Generation in 2D Space [60.46777591995821]
  We propose a novel method to convert 3D shapes into compact 1-channel geometry images and leverage StyleGAN3 and image-to-image translation networks to generate 3D objects in 2D space. The generated geometry images are quick to convert to 3D meshes, enabling real-time 3D object synthesis, visualization and interactive editing. We show both quantitatively and qualitatively that our method is highly effective at various tasks such as 3D shape generation, single view reconstruction and shape manipulation, while being significantly faster and more flexible compared to recent 3D generative models.
  arXiv  Detail & Related papers  (2022-10-06T15:54:01Z)
• GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images [72.15855070133425]
  We introduce GET3D, a Generative model that directly generates Explicit Textured 3D meshes with complex topology, rich geometric details, and high-fidelity textures. GET3D is able to generate high-quality 3D textured meshes, ranging from cars, chairs, animals, motorbikes and human characters to buildings.
  arXiv  Detail & Related papers  (2022-09-22T17:16:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.