Make-A-Character: High Quality Text-to-3D Character Generation within Minutes

• URL: http://arxiv.org/abs/2312.15430v1
• Date: Sun, 24 Dec 2023 08:11:39 GMT
• Title: Make-A-Character: High Quality Text-to-3D Character Generation within Minutes
• Authors: Jianqiang Ren, Chao He, Lin Liu, Jiahao Chen, Yutong Wang, Yafei Song, Jianfang Li, Tangli Xue, Siqi Hu, Tao Chen, Kunkun Zheng, Jianjing Xiang, Liefeng Bo
• Abstract summary: We propose a user-friendly framework named Make-A-Character (Mach) to create lifelike 3D avatars from text descriptions. Our system offers an intuitive approach for users to craft controllable, realistic, fully-realized 3D characters that meet their expectations within 2 minutes.
• Score: 28.30067870250642
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There is a growing demand for customized and expressive 3D characters with the emergence of AI agents and Metaverse, but creating 3D characters using traditional computer graphics tools is a complex and time-consuming task. To address these challenges, we propose a user-friendly framework named Make-A-Character (Mach) to create lifelike 3D avatars from text descriptions. The framework leverages the power of large language and vision models for textual intention understanding and intermediate image generation, followed by a series of human-oriented visual perception and 3D generation modules. Our system offers an intuitive approach for users to craft controllable, realistic, fully-realized 3D characters that meet their expectations within 2 minutes, while also enabling easy integration with existing CG pipeline for dynamic expressiveness. For more information, please visit the project page at https://human3daigc.github.io/MACH/.

Related papers

• Story3D-Agent: Exploring 3D Storytelling Visualization with Large Language Models [57.30913211264333]
  We present Story3D-Agent, a pioneering approach that transforms provided narratives into 3D-rendered visualizations. By integrating procedural modeling, our approach enables precise control over multi-character actions and motions, as well as diverse decorative elements. We have thoroughly evaluated our Story3D-Agent to validate its effectiveness, offering a basic framework to advance 3D story representation.
  arXiv  Detail & Related papers  (2024-08-21T17:43:15Z)
• SceneTeller: Language-to-3D Scene Generation [15.209079637302905]
  Given a prompt in natural language describing the object placement in the room, our method produces a high-quality 3D scene corresponding to it. Our turnkey pipeline produces state-of-the-art 3D scenes, while being easy to use even for novices.
  arXiv  Detail & Related papers  (2024-07-30T10:45:28Z)
• Magic3DSketch: Create Colorful 3D Models From Sketch-Based 3D Modeling Guided by Text and Language-Image Pre-Training [2.9600148687385786]
  Traditional methods like Computer-Aided Design (CAD) are often too labor-intensive and skill-demanding, making it challenging for novice users. Our proposed method, Magic3DSketch, employs a novel technique that encodes sketches to predict a 3D mesh, guided by text descriptions. Our method is also more useful and offers higher degree of controllability compared to existing text-to-3D approaches.
  arXiv  Detail & Related papers  (2024-07-27T09:59:13Z)
• Chat-Edit-3D: Interactive 3D Scene Editing via Text Prompts [76.73043724587679]
  We propose a dialogue-based 3D scene editing approach, termed CE3D. Hash-Atlas represents 3D scene views, which transfers the editing of 3D scenes onto 2D atlas images. Results demonstrate that CE3D effectively integrates multiple visual models to achieve diverse editing visual effects.
  arXiv  Detail & Related papers  (2024-07-09T13:24:42Z)
• Meta 3D Gen [57.313835190702484]
  3DGen offers 3D asset creation with high prompt fidelity and high-quality 3D shapes and textures in under a minute. 3DGen supports physically-based rendering (PBR), necessary for 3D asset relighting in real-world applications.
  arXiv  Detail & Related papers  (2024-07-02T18:37:52Z)
• Agent3D-Zero: An Agent for Zero-shot 3D Understanding [79.88440434836673]
  Agent3D-Zero is an innovative 3D-aware agent framework addressing the 3D scene understanding. We propose a novel way to make use of a Large Visual Language Model (VLM) via actively selecting and analyzing a series of viewpoints for 3D understanding. A distinctive advantage of Agent3D-Zero is the introduction of novel visual prompts, which significantly unleash the VLMs' ability to identify the most informative viewpoints.
  arXiv  Detail & Related papers  (2024-03-18T14:47:03Z)
• IPDreamer: Appearance-Controllable 3D Object Generation with Complex Image Prompts [90.49024750432139]
  We present IPDreamer, a novel method that captures intricate appearance features from complex $textbfI$mage $textbfP$rompts and aligns the synthesized 3D object with these extracted features. Our experiments demonstrate that IPDreamer consistently generates high-quality 3D objects that align with both the textual and complex image prompts.
  arXiv  Detail & Related papers  (2023-10-09T03:11:08Z)
• TADA! Text to Animatable Digital Avatars [57.52707683788961]
  TADA takes textual descriptions and produces expressive 3D avatars with high-quality geometry and lifelike textures. We derive an optimizable high-resolution body model from SMPL-X with 3D displacements and a texture map. We render normals and RGB images of the generated character and exploit their latent embeddings in the SDS training process.
  arXiv  Detail & Related papers  (2023-08-21T17:59:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.