3D-CoS: A New 3D Reconstruction Paradigm Based on VLM Code Synthesis

• URL: http://arxiv.org/abs/2606.10478v1
• Date: Tue, 09 Jun 2026 06:46:29 GMT
• Title: 3D-CoS: A New 3D Reconstruction Paradigm Based on VLM Code Synthesis
• Abstract summary: We propose and systematically evaluate a new 3D reconstruction paradigm, where 3D assets are constructed as executable code.<n>Our study shows that code as a 3D representation offers strong controllability and locality, yielding stronger edit fidelity and better preservation of unedited regions.
• Score: 69.23609431485401
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most recent 3D reconstruction and editing systems operate on implicit and explicit representations such as NeRF, point clouds, or meshes. While these representations enable high-fidelity rendering, they are fundamentally low-level and hard to control programmatically. In contrast, we propose and systematically evaluate a new 3D reconstruction paradigm, 3D Code Synthesis (3D-CoS), where 3D assets are constructed as executable Blender code, a programmatic and interpretable medium. To assess how well current VLMs can use code to represent 3D objects, we evaluate representative open-source and closed-source VLMs in code-based reconstruction under a unified protocol. We further introduce a suite of structured code-synthesis workflows, including blueprint-based planning, Retrieval-Augmented Generation (RAG) over Blender API documentation, few-shot geometric demonstrations, and a component-level Agent workflow for part-wise code generation. To demonstrate the unique advantages of this representation, we further evaluate localized text-driven modifications and compare our code-based edits with a point-cloud-based 3D editing baseline. Our study shows that code as a 3D representation offers strong controllability and locality, yielding stronger edit fidelity and better preservation of unedited regions in our targeted editing evaluation. Our work also analyzes the potential of this paradigm, delineates the current capability frontier of VLMs for programmatic 3D modeling, and highlights code synthesis as a promising direction for editable 3D reconstruction.

Related papers

• Procedura: Agentic 3D Modeling with Procedural Control [52.232812096564906]
  Procedura is a novel 3D modeling agent framework that writes an object as a procedural assembly.<n>From a text prompt, the agent plans the object as an assembly graph and writes the program part by part.<n>A decoupled vision critic then refines the assembly one diagnosed fix at a time.
  arXiv  Detail & Related papers  (2026-08-26T17:54:39Z)
• 3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code [25.904997126319547]
  3DCodeBench is a systematic benchmark for evaluating vision-language model (VLM) agents for procedural 3D generation in 3D modeling software.<n>We build 3DCodeArena, a ranking platform based on pairwise human preferences over generated 3D outputs.
  arXiv  Detail & Related papers  (2026-05-31T06:59:49Z)
• Code-as-Room: Generating 3D Rooms from Top-Down View Images via Agentic Code Synthesis [12.68633443613779]
  We propose Code-as-Room, an MLLM-based agentic framework equipped with a structured execution harness.<n>Given a top-down room image, the framework parses the reference image to extract scene elements and their spatial relationships.<n>A cross-stage memory module is maintained throughout to mitigate context inherent to existing agent-based frameworks.
  arXiv  Detail & Related papers  (2026-05-18T14:18:36Z)
• AREA3D: Active Reconstruction Agent with Unified Feed-Forward 3D Perception and Vision-Language Guidance [36.125573065910594]
  Active 3D reconstruction enables an agent to autonomously select viewpoints to obtain accurate and complete scene geometry.<n>We propose AREA3D, an active reconstruction agent that leverages feed-forward 3D reconstruction models and vision-language guidance.<n>Our framework decouples view-uncertainty modeling from the underlying feed-forward reconstructor, enabling precise uncertainty estimation without expensive online optimization.
  arXiv  Detail & Related papers  (2025-11-28T06:17:02Z)
• EA3D: Online Open-World 3D Object Extraction from Streaming Videos [55.48835711373918]
  We present ExtractAnything3D (EA3D), a unified online framework for open-world 3D object extraction.<n>Given a streaming video, EA3D dynamically interprets each frame using vision-language and 2D vision foundation encoders to extract object-level knowledge.<n>A recurrent joint optimization module directs the model's attention to regions of interest, simultaneously enhancing both geometric reconstruction and semantic understanding.
  arXiv  Detail & Related papers  (2025-10-29T03:56:41Z)
• Unifi3D: A Study on 3D Representations for Generation and Reconstruction in a Common Framework [12.995265721478132]
  We present a unified evaluation framework designed to assess the performance of 3D representations in reconstruction and generation.<n>We derive best practices over all steps involved in the 3D generation pipeline, including preprocessing, mesh reconstruction, compression with autoencoders, and generation.
  arXiv  Detail & Related papers  (2025-09-02T16:25:12Z)
• MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Clouds [50.98900790623827]
  MeshCoder is a novel framework that reconstructs complex 3D objects from point clouds into editable Blender Python scripts.<n>We train a multimodal large language model (LLM) that translates 3D point cloud into executable Blender Python scripts.<n>Our approach achieves superior performance in shape-to-code reconstruction tasks and also facilitates intuitive geometric and topological editing.
  arXiv  Detail & Related papers  (2025-08-20T17:50:15Z)
• Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Survey [171.72616707259306]
  3D reconstruction and view synthesis are foundational problems in computer vision, graphics, and immersive technologies such as augmented reality (AR), virtual reality (VR), and digital twins.<n>Recent advances in feed-forward approaches, driven by deep learning, have revolutionized this field by enabling fast and generalizable 3D reconstruction and view synthesis.
  arXiv  Detail & Related papers  (2025-07-19T06:13:25Z)
• TAR3D: Creating High-Quality 3D Assets via Next-Part Prediction [137.34863114016483]
  TAR3D is a novel framework that consists of a 3D-aware Vector Quantized-Variational AutoEncoder (VQ-VAE) and a Generative Pre-trained Transformer (GPT)<n>We show that TAR3D can achieve superior generation quality over existing methods in text-to-3D and image-to-3D tasks.
  arXiv  Detail & Related papers  (2024-12-22T08:28:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.