AI-powered Contextual 3D Environment Generation: A Systematic Review

• URL: http://arxiv.org/abs/2506.05449v1
• Date: Thu, 05 Jun 2025 15:56:28 GMT
• Title: AI-powered Contextual 3D Environment Generation: A Systematic Review
• Authors: Miguel Silva, Alexandre Valle de Carvalho,
• Abstract summary: This study performs a systematic review of existing generative AI techniques for 3D scene generation.<n>By examining state-of-the-art approaches, it presents key challenges such as scene authenticity and the influence of textual inputs.
• Score: 49.1574468325115
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The generation of high-quality 3D environments is crucial for industries such as gaming, virtual reality, and cinema, yet remains resource-intensive due to the reliance on manual processes. This study performs a systematic review of existing generative AI techniques for 3D scene generation, analyzing their characteristics, strengths, limitations, and potential for improvement. By examining state-of-the-art approaches, it presents key challenges such as scene authenticity and the influence of textual inputs. Special attention is given to how AI can blend different stylistic domains while maintaining coherence, the impact of training data on output quality, and the limitations of current models. In addition, this review surveys existing evaluation metrics for assessing realism and explores how industry professionals incorporate AI into their workflows. The findings of this study aim to provide a comprehensive understanding of the current landscape and serve as a foundation for future research on AI-driven 3D content generation. Key findings include that advanced generative architectures enable high-quality 3D content creation at a high computational cost, effective multi-modal integration techniques like cross-attention and latent space alignment facilitate text-to-3D tasks, and the quality and diversity of training data combined with comprehensive evaluation metrics are critical to achieving scalable, robust 3D scene generation.

Related papers

• Hi3DEval: Advancing 3D Generation Evaluation with Hierarchical Validity [78.7107376451476]
  Hi3DEval is a hierarchical evaluation framework tailored for 3D generative content.<n>We extend texture evaluation beyond aesthetic appearance by explicitly assessing material realism.<n>We propose a 3D-aware automated scoring system based on hybrid 3D representations.
  arXiv  Detail & Related papers  (2025-08-07T17:50:13Z)
• Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Survey [154.50661618628433]
  3D reconstruction and view synthesis are foundational problems in computer vision, graphics, and immersive technologies such as augmented reality (AR), virtual reality (VR), and digital twins.<n>Recent advances in feed-forward approaches, driven by deep learning, have revolutionized this field by enabling fast and generalizable 3D reconstruction and view synthesis.
  arXiv  Detail & Related papers  (2025-07-19T06:13:25Z)
• Towards Depth Foundation Model: Recent Trends in Vision-Based Depth Estimation [75.30238170051291]
  Depth estimation is a fundamental task in 3D computer vision, crucial for applications such as 3D reconstruction, free-viewpoint rendering, robotics, autonomous driving, and AR/VR technologies.<n>Traditional methods relying on hardware sensors like LiDAR are often limited by high costs, low resolution, and environmental sensitivity, limiting their applicability in real-world scenarios.<n>Recent advances in vision-based methods offer a promising alternative, yet they face challenges in generalization and stability due to either the low-capacity model architectures or the reliance on domain-specific and small-scale datasets.
  arXiv  Detail & Related papers  (2025-07-15T17:59:59Z)
• MetaScenes: Towards Automated Replica Creation for Real-world 3D Scans [76.39726619818896]
  Embodied AI (EAI) research requires high-quality, diverse 3D scenes to support skill acquisition, sim-to-real transfer, and generalization.<n>Existing datasets demonstrate that this process heavily relies on artist-driven designs.<n>We present MetaScenes, a large-scale, simulatable 3D scene dataset constructed from real-world scans.
  arXiv  Detail & Related papers  (2025-05-05T06:13:25Z)
• Recent Advance in 3D Object and Scene Generation: A Survey [14.673302810271219]
  This survey aims to provide readers with a structured understanding of state-of-the-art 3D generation technologies.<n>We focus on three dominant paradigms: layout-guided compositional synthesis, 2D prior-based scene generation, and rule-driven modeling.
  arXiv  Detail & Related papers  (2025-04-16T03:22:06Z)
• Embodied Intelligence for 3D Understanding: A Survey on 3D Scene Question Answering [28.717312557697376]
  3D Scene Question Answering represents an interdisciplinary task that integrates 3D visual perception and natural language processing.<n>Recent advances in large multimodal modelling have driven the creation of diverse datasets and spurred the development of instruction-tuning and zero-shot methods for 3D SQA.<n>This paper presents the first comprehensive survey of 3D SQA, systematically reviewing datasets, methodologies, and evaluation metrics.
  arXiv  Detail & Related papers  (2025-02-01T07:01:33Z)
• 3DGCQA: A Quality Assessment Database for 3D AI-Generated Contents [50.730468291265886]
  This paper introduces a novel 3DGC quality assessment dataset, 3DGCQA, built using 7 representative Text-to-3D generation methods. The visualization intuitively reveals the presence of 6 common distortion categories in the generated 3DGCs. subjective quality assessment is conducted by evaluators, whose ratings reveal significant variation in quality across different generation methods. Several objective quality assessment algorithms are tested on the 3DGCQA dataset.
  arXiv  Detail & Related papers  (2024-09-11T12:47:40Z)
• Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding [50.448520056844885]
  We propose a generative Bayesian network to produce diverse synthetic scenes with real-world patterns. A series of experiments robustly display our method's consistent superiority over existing state-of-the-art pre-training approaches.
  arXiv  Detail & Related papers  (2024-06-17T07:43:53Z)
• A Survey On Text-to-3D Contents Generation In The Wild [5.875257756382124]
  3D content creation plays a vital role in various applications, such as gaming, robotics simulation, and virtual reality. To address this challenge, text-to-3D generation technologies have emerged as a promising solution for automating 3D creation.
  arXiv  Detail & Related papers  (2024-05-15T15:23:22Z)
• A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes [80.20670062509723]
  3D dense captioning is an emerging vision-language bridging task that aims to generate detailed descriptions for 3D scenes. It presents significant potential and challenges due to its closer representation of the real world compared to 2D visual captioning. Despite the popularity and success of existing methods, there is a lack of comprehensive surveys summarizing the advancements in this field.
  arXiv  Detail & Related papers  (2024-03-12T10:04:08Z)
• 3D objects and scenes classification, recognition, segmentation, and reconstruction using 3D point cloud data: A review [5.85206759397617]
  Three-dimensional (3D) point cloud analysis has become one of the attractive subjects in realistic imaging and machine visions. A significant effort has recently been devoted to developing novel strategies, using different techniques such as deep learning models. Various tasks performed on 3D point could data are investigated, including objects and scenes detection, recognition, segmentation and reconstruction.
  arXiv  Detail & Related papers  (2023-06-09T15:45:23Z)
• Joint Supervised and Self-Supervised Learning for 3D Real-World Challenges [16.328866317851187]
  Point cloud processing and 3D shape understanding are challenging tasks for which deep learning techniques have demonstrated great potentials. Here we consider several possible scenarios involving synthetic and real-world point clouds where supervised learning fails due to data scarcity and large domain gaps. We propose to enrich standard feature representations by leveraging self-supervision through a multi-task model that can solve a 3D puzzle while learning the main task of shape classification or part segmentation.
  arXiv  Detail & Related papers  (2020-04-15T23:34:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.