Advances in 4D Generation: A Survey

• URL: http://arxiv.org/abs/2503.14501v2
• Date: Wed, 19 Mar 2025 08:05:50 GMT
• Title: Advances in 4D Generation: A Survey
• Authors: Qiaowei Miao, Kehan Li, Jinsheng Quan, Zhiyuan Min, Shaojie Ma, Yichao Xu, Yi Yang, Yawei Luo,
• Abstract summary: 4D generation focuses on creating dynamic 3D assets with consistency based on user input.<n>We summarize five major challenges of 4D generation: consistency, controllability, diversity, efficiency, and fidelity.<n>We provide an in-depth discussion of the obstacles currently hindering the development of the 4D generation.
• Score: 20.285058992203442
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative artificial intelligence (AI) has made significant progress across various domains in recent years. Building on the rapid advancements in 2D, video, and 3D content generation fields, 4D generation has emerged as a novel and rapidly evolving research area, attracting growing attention. 4D generation focuses on creating dynamic 3D assets with spatiotemporal consistency based on user input, offering greater creative freedom and richer immersive experiences. This paper presents a comprehensive survey of the 4D generation field, systematically summarizing its core technologies, developmental trajectory, key challenges, and practical applications, while also exploring potential future research directions. The survey begins by introducing various fundamental 4D representation models, followed by a review of 4D generation frameworks built upon these representations and the key technologies that incorporate motion and geometry priors into 4D assets. We summarize five major challenges of 4D generation: consistency, controllability, diversity, efficiency, and fidelity, accompanied by an outline of existing solutions to address these issues. We systematically analyze applications of 4D generation, spanning dynamic object generation, scene generation, digital human synthesis, 4D editing, and autonomous driving. Finally, we provide an in-depth discussion of the obstacles currently hindering the development of the 4D generation. This survey offers a clear and comprehensive overview of 4D generation, aiming to stimulate further exploration and innovation in this rapidly evolving field. Our code is publicly available at: https://github.com/MiaoQiaowei/Awesome-4D.

Related papers

• Learning 4D Panoptic Scene Graph Generation from Rich 2D Visual Scene [122.42861221739123]
  This paper investigates a novel framework for 4D-PSG generation that leverages rich 2D visual scene annotations to enhance 4D scene learning. We propose a 2D-to-4D visual scene transfer learning framework, where a spatial-temporal scene strategy effectively transfers dimension-invariant features from abundant 2D SG annotations to 4D scenes.
  arXiv  Detail & Related papers  (2025-03-19T09:16:08Z)
• WideRange4D: Enabling High-Quality 4D Reconstruction with Wide-Range Movements and Scenes [65.76371201992654]
  We propose a novel 4D reconstruction benchmark, WideRange4D.<n>This benchmark includes rich 4D scene data with large spatial variations, allowing for a more comprehensive evaluation of the generation capabilities of 4D generation methods.<n>We also introduce a new 4D reconstruction method, Progress4D, which generates stable and high-quality 4D results across various complex 4D scene reconstruction tasks.
  arXiv  Detail & Related papers  (2025-03-17T17:58:18Z)
• Simulating the Real World: A Unified Survey of Multimodal Generative Models [48.35284571052435]
  We present a unified survey for multimodal generative models that investigate the progression of data dimensionality in real-world simulation.<n>To the best of our knowledge, this is the first attempt to systematically unify the study of 2D, video, 3D and 4D generation within a single framework.
  arXiv  Detail & Related papers  (2025-03-06T17:31:43Z)
• Dynamic Realms: 4D Content Analysis, Recovery and Generation with Geometric, Topological and Physical Priors [0.8339831319589133]
  My research focuses on the analysis, recovery, and generation of 4D content, where 4D includes three spatial dimensions (x, y, z) and a temporal dimension t, such as shape and motion. My research aims to make 4D content generation more efficient, accessible, and higher in quality by incorporating geometric, topological, and physical priors.
  arXiv  Detail & Related papers  (2024-09-23T03:46:51Z)
• Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Models [116.31344506738816]
  We present a novel framework, textbfDiffusion4D, for efficient and scalable 4D content generation. We develop a 4D-aware video diffusion model capable of synthesizing orbital views of dynamic 3D assets. Our method surpasses prior state-of-the-art techniques in terms of generation efficiency and 4D geometry consistency.
  arXiv  Detail & Related papers  (2024-05-26T17:47:34Z)
• A Survey On Text-to-3D Contents Generation In The Wild [5.875257756382124]
  3D content creation plays a vital role in various applications, such as gaming, robotics simulation, and virtual reality. To address this challenge, text-to-3D generation technologies have emerged as a promising solution for automating 3D creation.
  arXiv  Detail & Related papers  (2024-05-15T15:23:22Z)
• Comp4D: LLM-Guided Compositional 4D Scene Generation [65.5810466788355]
  We present Comp4D, a novel framework for Compositional 4D Generation. Unlike conventional methods that generate a singular 4D representation of the entire scene, Comp4D innovatively constructs each 4D object within the scene separately. Our method employs a compositional score distillation technique guided by the pre-defined trajectories.
  arXiv  Detail & Related papers  (2024-03-25T17:55:52Z)
• A Comprehensive Survey on 3D Content Generation [148.434661725242]
  3D content generation shows both academic and practical values. New taxonomy is proposed that categorizes existing approaches into three types: 3D native generative methods, 2D prior-based 3D generative methods, and hybrid 3D generative methods.
  arXiv  Detail & Related papers  (2024-02-02T06:20:44Z)
• Advances in 3D Generation: A Survey [54.95024616672868]
  The field of 3D content generation is developing rapidly, enabling the creation of increasingly high-quality and diverse 3D models. Specifically, we introduce the 3D representations that serve as the backbone for 3D generation. We provide a comprehensive overview of the rapidly growing literature on generation methods, categorized by the type of algorithmic paradigms.
  arXiv  Detail & Related papers  (2024-01-31T13:06:48Z)
• 4DGen: Grounded 4D Content Generation with Spatial-temporal Consistency [118.15258850780417]
  We present textbf4DGen, a novel framework for grounded 4D content creation. Our pipeline facilitates controllable 4D generation, enabling users to specify the motion via monocular video or adopt image-to-video generations. Compared to existing video-to-4D baselines, our approach yields superior results in faithfully reconstructing input signals.
  arXiv  Detail & Related papers  (2023-12-28T18:53:39Z)
• Generative AI meets 3D: A Survey on Text-to-3D in AIGC Era [36.66506237523448]
  Generative AI has made significant progress in recent years, with text-guided content generation being the most practical. Thanks to advancements in text-to-image and 3D modeling technologies, like neural radiance field (NeRF), text-to-3D has emerged as a nascent yet highly active research field.
  arXiv  Detail & Related papers  (2023-05-10T13:26:08Z)
• Towards AI-Architecture Liberty: A Comprehensive Survey on Design and Generation of Virtual Architecture by Deep Learning [23.58793497403681]
  3D shape generation techniques leveraging deep learning have garnered significant interest from both the computer vision and architectural design communities. We review 149 related articles covering architectural design, 3D shape techniques, and virtual environments. We highlight four important enablers of ubiquitous interaction with immersive systems in deep learning-assisted architectural generation.
  arXiv  Detail & Related papers  (2023-04-30T15:38:36Z)
• LoRD: Local 4D Implicit Representation for High-Fidelity Dynamic Human Modeling [69.56581851211841]
  We propose a novel Local 4D implicit Representation for Dynamic clothed human, named LoRD. Our key insight is to encourage the network to learn the latent codes of local part-level representation. LoRD has strong capability for representing 4D human, and outperforms state-of-the-art methods on practical applications.
  arXiv  Detail & Related papers  (2022-08-18T03:49:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.