LL3M: Large Language 3D Modelers

• URL: http://arxiv.org/abs/2508.08228v1
• Date: Mon, 11 Aug 2025 17:48:02 GMT
• Title: LL3M: Large Language 3D Modelers
• Authors: Sining Lu, Guan Chen, Nam Anh Dinh, Itai Lang, Ari Holtzman, Rana Hanocka,
• Abstract summary: We present LL3M, a system that generates 3D assets by writing interpretable Python code in Blender.<n>We reformulate shape generation as a code-writing task, enabling greater modularity, editability, and integration with artist Blender.<n>Our experiments showcase the power of code as a generative and interpretable medium for 3D asset creation.
• Score: 18.23329430829059
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present LL3M, a multi-agent system that leverages pretrained large language models (LLMs) to generate 3D assets by writing interpretable Python code in Blender. We break away from the typical generative approach that learns from a collection of 3D data. Instead, we reformulate shape generation as a code-writing task, enabling greater modularity, editability, and integration with artist workflows. Given a text prompt, LL3M coordinates a team of specialized LLM agents to plan, retrieve, write, debug, and refine Blender scripts that generate and edit geometry and appearance. The generated code works as a high-level, interpretable, human-readable, well-documented representation of scenes and objects, making full use of sophisticated Blender constructs (e.g. B-meshes, geometry modifiers, shader nodes) for diverse, unconstrained shapes, materials, and scenes. This code presents many avenues for further agent and human editing and experimentation via code tweaks or procedural parameters. This medium naturally enables a co-creative loop in our system: agents can automatically self-critique using code and visuals, while iterative user instructions provide an intuitive way to refine assets. A shared code context across agents enables awareness of previous attempts, and a retrieval-augmented generation knowledge base built from Blender API documentation, BlenderRAG, equips agents with examples, types, and functions empowering advanced modeling operations and code correctness. We demonstrate the effectiveness of LL3M across diverse shape categories, style and material edits, and user-driven refinements. Our experiments showcase the power of code as a generative and interpretable medium for 3D asset creation. Our project page is at https://threedle.github.io/ll3m.

Related papers

• ShapeCraft: LLM Agents for Structured, Textured and Interactive 3D Modeling [45.20958044544727]
  ShapeCraft is a novel multi-agent framework for text-to-3D generation.<n>ShapeCraft decomposes complex natural language into a structured graph of sub-tasks.<n>Agents parse user input to GPS, then iteratively refine procedural modeling and painting to produce 3D assets.
  arXiv  Detail & Related papers  (2025-10-20T14:51:14Z)
• MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Clouds [50.98900790623827]
  MeshCoder is a novel framework that reconstructs complex 3D objects from point clouds into editable Blender Python scripts.<n>We train a multimodal large language model (LLM) that translates 3D point cloud into executable Blender Python scripts.<n>Our approach achieves superior performance in shape-to-code reconstruction tasks and also facilitates intuitive geometric and topological editing.
  arXiv  Detail & Related papers  (2025-08-20T17:50:15Z)
• Cube: A Roblox View of 3D Intelligence [67.43543266278154]
  Foundation models trained on vast amounts of data have demonstrated remarkable reasoning and generation capabilities.<n>We show how our tokenization scheme can be used in applications for text-to-shape generation, shape-to-text generation and text-to-scene generation.<n>We conclude with a discussion outlining our path to building a fully unified foundation model for 3D intelligence.
  arXiv  Detail & Related papers  (2025-03-19T17:52:17Z)
• Instructive3D: Editing Large Reconstruction Models with Text Instructions [2.9575146209034853]
  Instructive3D is a novel LRM based model that integrates generation and fine-grained editing, through user text prompts, of 3D objects into a single model.<n>We find that Instructive3D produces superior 3D objects with the properties specified by the edit prompts.
  arXiv  Detail & Related papers  (2025-01-08T09:28:25Z)
• PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Models [63.1432721793683]
  We introduce PartGen, a novel approach that generates 3D objects composed of meaningful parts starting from text, an image, or an unstructured 3D object.<n>We evaluate our method on generated and real 3D assets and show that it outperforms segmentation and part-extraction baselines by a large margin.
  arXiv  Detail & Related papers  (2024-12-24T18:59:43Z)
• Make-it-Real: Unleashing Large Multimodal Model for Painting 3D Objects with Realistic Materials [108.59709545364395]
  GPT-4V can effectively recognize and describe materials, allowing the construction of a detailed material library. The correctly matched materials are then meticulously applied as reference for the new SVBRDF material generation. Make-it-Real offers a streamlined integration into the 3D content creation workflow.
  arXiv  Detail & Related papers  (2024-04-25T17:59:58Z)
• GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting [52.150502668874495]
  We present GALA3D, generative 3D GAussians with LAyout-guided control, for effective compositional text-to-3D generation. GALA3D is a user-friendly, end-to-end framework for state-of-the-art scene-level 3D content generation and controllable editing.
  arXiv  Detail & Related papers  (2024-02-11T13:40:08Z)
• 3D-GPT: Procedural 3D Modeling with Large Language Models [47.72968643115063]
  We introduce 3D-GPT, a framework utilizing large language models(LLMs) for instruction-driven 3D modeling. 3D-GPT positions LLMs as proficient problem solvers, dissecting the procedural 3D modeling tasks into accessible segments and appointing the apt agent for each task. Our empirical investigations confirm that 3D-GPT not only interprets and executes instructions, delivering reliable results but also collaborates effectively with human designers.
  arXiv  Detail & Related papers  (2023-10-19T17:41:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.