CoGen3D: An Agentic Human-AI Co-Design Pipeline for 3D Asset Generation for Virtual Reality

• URL: http://arxiv.org/abs/2607.03731v1
• Date: Sat, 04 Jul 2026 06:37:26 GMT
• Title: CoGen3D: An Agentic Human-AI Co-Design Pipeline for 3D Asset Generation for Virtual Reality
• Abstract summary: CoGen3D proactively guides users through conversational intent elicitation, a concept image confirmation, and image-to-3D generation.<n>Our results suggest that our staged, intent-based co-design can democratize virtual reality authoring.
• Score: 24.365227177031098
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Creating 3D assets for virtual reality requires modeling expertise, which restricts the authorship of immersive experiences. Existing generative AI tools rely on unconstrained, command-driven prompting, lacking the conversational scaffolding needed for users to articulate their intent and validate designs prior to rendering. To address this, we introduce CoGen3D, an agentic human-AI co-design pipeline that proactively guides users through conversational intent elicitation, a concept image confirmation, and image-to-3D generation that directly deploys to immersive scenes. We evaluated this system through a user study (N=120) across six affectively diverse immersive scenes, observing 60 Design group participants who co-created 3D assets for the scenes, and 60 Validation group participants who experienced the scenes with generated assets. Our findings show that co-designed assets are associated with higher scene engagement and shifted affective responses, while participants generally preferred concept images over the final 3D assets, with no increased leniency toward degradation in their own creations. Analysis of the human-AI conversations further shows that target environments shape users' conversational patterns. Our results suggest that our staged, intent-based co-design can democratize virtual reality authoring and shift immersive content creation from technical execution toward collaborative spatial design.

Related papers

• 3D Generation for Embodied AI and Robotic Simulation: A Survey [19.441445845534158]
  Embodied AI and robotic systems increasingly depend on scalable, diverse, and physically grounded 3D content for simulation-based training and real-world deployment.<n>This survey reviews 3D generation for embodied AI and organizes the literature around three roles that 3D generation plays in embodied systems.<n>In Data Generator, 3D generation produces simulation-ready objects and assets, including articulated, physically grounded, and deformable content for downstream interaction.<n>In Simulation Environments, it constructs interactive and task-oriented worlds, spanning structure-aware, controllable, and agentic scene generation.<n>In Sim2Real Bridge, it supports digital twin reconstruction
  arXiv  Detail & Related papers  (2026-04-29T10:17:55Z)
• SceneExpander: Expanding 3D Scenes with Free-Form Inserted Views [69.08965991211704]
  We study 3D scene expansion in a user-centric workflow, starting from a real scene captured by multi-view images.<n>Unlike simple object editing or style transfer in a fixed scene, the inserted view is often 3D-misaligned with the original reconstruction.<n>We propose SceneExpander, which applies test-time adaptation to a parametric feed-forward 3D reconstruction model.
  arXiv  Detail & Related papers  (2026-03-28T02:04:48Z)
• Hunyuan3D Studio: End-to-End AI Pipeline for Game-Ready 3D Asset Generation [85.68100076594197]
  Hunyuan3D Studio is an end-to-end AI-powered content creation platform designed to revolutionize the game production pipeline.<n>At its core, Hunyuan3D Studio integrates a suite of advanced neural modules into a cohesive and user-friendly system.
  arXiv  Detail & Related papers  (2025-09-16T08:33:03Z)
• SceneGen: Single-Image 3D Scene Generation in One Feedforward Pass [44.087747627571716]
  3D content generation has attracted significant research interest due to its applications in VR/AR and embodied AI.<n>We present SceneGen, a novel framework that takes a scene image and corresponding object masks as input, simultaneously producing multiple 3D assets.<n>We believe this paradigm offers a novel solution for high-quality 3D content generation, potentially advancing its practical applications in downstream tasks.
  arXiv  Detail & Related papers  (2025-08-21T17:59:16Z)
• HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation [50.206100327643284]
  HiScene is a novel hierarchical framework that bridges the gap between 2D image generation and 3D object generation.<n>We generate 3D content that aligns with 2D representations while maintaining compositional structure.
  arXiv  Detail & Related papers  (2025-04-17T16:33:39Z)
• WorldCraft: Photo-Realistic 3D World Creation and Customization via LLM Agents [67.31920821192323]
  We introduce WorldCraft, a system where large language model (LLM) agents leverage procedural generation to create scenes populated with objects.<n>In our framework, a coordinator agent manages the overall process and works with two specialized LLM agents to complete the scene creation.<n>Our pipeline incorporates a trajectory control agent, allowing users to animate the scene and operate the camera through natural language interactions.
  arXiv  Detail & Related papers  (2025-02-21T17:18:30Z)
• Generative AI Framework for 3D Object Generation in Augmented Reality [0.0]
  This thesis integrates state-of-the-art generative AI models for real-time creation of 3D objects in augmented reality (AR) environments.<n>The framework demonstrates applications across industries such as gaming, education, retail, and interior design.<n>A significant contribution is democratizing 3D model creation, making advanced AI tools accessible to a broader audience.
  arXiv  Detail & Related papers  (2025-02-21T17:01:48Z)
• SpaceBlender: Creating Context-Rich Collaborative Spaces Through Generative 3D Scene Blending [19.06858242647237]
  We introduce SpaceBlender, a pipeline that transforms user-provided 2D images into context-rich 3D environments. Participants appreciated the enhanced familiarity and context provided by SpaceBlender but noted complexities in the generative environments. We propose directions for improving the pipeline and discuss the value and design of blended spaces for different scenarios.
  arXiv  Detail & Related papers  (2024-09-20T22:27:31Z)
• iControl3D: An Interactive System for Controllable 3D Scene Generation [57.048647153684485]
  iControl3D is a novel interactive system that empowers users to generate and render customizable 3D scenes with precise control. We leverage 3D meshes as an intermediary proxy to iteratively merge individual 2D diffusion-generated images into a cohesive and unified 3D scene representation. Our neural rendering interface enables users to build a radiance field of their scene online and navigate the entire scene.
  arXiv  Detail & Related papers  (2024-08-03T06:35:09Z)
• I-Design: Personalized LLM Interior Designer [57.00412237555167]
  I-Design is a personalized interior designer that allows users to generate and visualize their design goals through natural language communication. I-Design starts with a team of large language model agents that engage in dialogues and logical reasoning with one another. The final design is then constructed in 3D by retrieving and integrating assets from an existing object database.
  arXiv  Detail & Related papers  (2024-04-03T16:17:53Z)
• Make-A-Character: High Quality Text-to-3D Character Generation within Minutes [28.30067870250642]
  We propose a user-friendly framework named Make-A-Character (Mach) to create lifelike 3D avatars from text descriptions. Our system offers an intuitive approach for users to craft controllable, realistic, fully-realized 3D characters that meet their expectations within 2 minutes.
  arXiv  Detail & Related papers  (2023-12-24T08:11:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.