Reflecting Process Expertise in Procedural Material Generation

• URL: http://arxiv.org/abs/2607.13318v2
• Date: Thu, 23 Jul 2026 10:02:57 GMT
• Title: Reflecting Process Expertise in Procedural Material Generation
• Abstract summary: Procedural material creation underpins applications in digital content creation, visual effects, and 3D asset design.<n>We represent expert synthesis as process traces: textual records of construction steps, parameters, and design intent.<n>We leverage tutorial videos as a source of process knowledge and extract textual traces using automated video analysis tools.
• Score: 47.15641864882028
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Procedural material creation underpins applications in digital content creation, visual effects, and 3D asset design. Achieving high-quality results requires more than reproducing node graphs -- it demands understanding the process by which experts construct materials. We formulate procedural material generation as retrieval-time process reasoning over expert demonstrations, elevating process to a first-class representation beyond graph-only synthesis. Concretely, we represent expert workflows as process traces: textual records of construction steps, parameters, and design intent. To instantiate this idea, we use a pretrained LLM-based ProcessSynthesizer to synthesize a process trace aligned with a user's intent and a pretrained LLM-based Compiler to ground the process trace into an executable Blender material graph. Because procedural expertise is most naturally conveyed through demonstrations, we leverage tutorial videos as a source of process knowledge and extract textual, LLM-compatible traces using automated video analysis tools. In an expert study with five Blender artists (avg. 7.5 years of experience), materials generated by reflecting expert demonstrations were found to produce workflows requiring fewer edits, and more closely match professional design strategies than methods operating solely on static artifacts. A user study with 150 participants further shows that our approach achieves superior generation and editing performance compared to prior procedural systems. All code, models, and data will be available at https://materialapprentice.github.io

Related papers

• Recovering Software Architecture Intent from Historical Work Items using Generative AI: A Mixed-Methods Industry Case Study [2.5501791028999583]
  This work investigates the recovery of C4 architecture diagrams from historical agile work items using an LLM-based pipeline.<n>The semi-automatic five-step workflow employs a prompt chain, bidirectional traceability, and Chain-of-Thought reasoning.<n>The proposed workflow demonstrates the practical viability of LLM-assisted architectural recovery based on development process artifacts.
  arXiv  Detail & Related papers  (2026-08-28T14:56:00Z)
• AI-based worker guidance in assembly and disassembly operations using multimodal ego/exo-centric data capture and structured task knowledge [9.167082845109439]
  Assembly and disassembly processes rely on expert knowledge that is difficult to document, reuse, and transfer.<n>This paper presents a data-centric approach for extracting structured task knowledge from expert demonstrations using egocentric and exocentric recordings.
  arXiv  Detail & Related papers  (2026-08-23T21:41:33Z)
• Demo2Tutorial: From Human Experience to Multimodal Software Tutorials [63.678709079907684]
  Demo2Tutorial is a framework that transforms experience captured via screen recordings and interaction logs into structured, multimodal software tutorials.<n>A Step Planner then abstracts these steps into hierarchical task graphs representing goals and steps.<n>A Composer transforms the parsed experience into structured, reusable image-text instructions.
  arXiv  Detail & Related papers  (2026-06-02T17:39:07Z)
• Towards Agentic Intelligence for Materials Science [73.4576385477731]
  This survey advances a unique pipeline-centric view that spans from corpus curation and pretraining to goal-conditioned agents interfacing with simulation and experimental platforms.<n>To bridge communities and establish a shared frame of reference, we first present an integrated lens that aligns terminology, evaluation, and workflow stages across AI and materials science.
  arXiv  Detail & Related papers  (2026-01-29T23:48:43Z)
• Large Language Model Agent for User-friendly Chemical Process Simulations [0.0]
  A large language model (LLM) agent is integrated with AVEVA Process Model Protocol (MCP), allowing natural language simulations.<n>Two case studies assess the framework across different task complexities and interaction modes.<n>The framework benefits both educational purposes, by translating technical concepts and demonstrating, and experienced practitioners by automating data extraction, speeding routine tasks, and supporting.<n>While current limitations such as oversimplification, calculation errors, and technical hiccups mean expert oversight is still needed, the framework suggests LLM-based agents can become valuable collaborators.
  arXiv  Detail & Related papers  (2026-01-15T12:18:45Z)
• VLMaterial: Procedural Material Generation with Large Vision-Language Models [43.60515109432699]
  We leverage the ability to convert procedural materials into standard Python programs.<n>We fine-tune a large pre-trained vision-language model to generate such programs from input images.<n>Our method outperforms previous methods on both synthetic and real-world examples.
  arXiv  Detail & Related papers  (2025-01-27T00:20:48Z)
• Large Language Models as Realistic Microservice Trace Generators [48.730974361862366]
  This paper proposes a first-of-a-kind approach that relies on training a large language model (LLM) to generate synthetic workload traces.<n>We show that TraceLLM produces diverse, realistic traces under varied conditions, outperforming existing approaches in both accuracy and validity.<n>TraceLLM adapts to downstream trace-related tasks, such as predicting key trace features and infilling missing data.
  arXiv  Detail & Related papers  (2024-12-16T12:48:04Z)
• AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials [53.376263056033046]
  Existing approaches rely on expensive human annotation, making them unsustainable at scale.<n>We propose AgentTrek, a scalable data synthesis pipeline that generates web agent trajectories by leveraging publicly available tutorials.<n>Our fully automated approach significantly reduces data collection costs, achieving a cost of just $0.55 per high-quality trajectory without human annotators.
  arXiv  Detail & Related papers  (2024-12-12T18:59:27Z)
• Chain of Thought Imitation with Procedure Cloning [129.62135987416164]
  We propose procedure cloning, which applies supervised sequence prediction to imitate the series of expert computations. We show that imitating the intermediate computations of an expert's behavior enables procedure cloning to learn policies exhibiting significant generalization to unseen environment configurations.
  arXiv  Detail & Related papers  (2022-05-22T13:14:09Z)
• Shedding Light on Blind Spots: Developing a Reference Architecture to Leverage Video Data for Process Mining [0.0]
  We propose a reference architecture to bridge the gap between computer vision and process mining. A prototype instantiation of the proposed reference architecture is capable of automatically extracting most of the process-relevant events from unstructured video data.
  arXiv  Detail & Related papers  (2020-10-21T20:01:52Z)
• Knowledge-Aware Procedural Text Understanding with Multi-Stage Training [110.93934567725826]
  We focus on the task of procedural text understanding, which aims to comprehend such documents and track entities' states and locations during a process. Two challenges, the difficulty of commonsense reasoning and data insufficiency, still remain unsolved. We propose a novel KnOwledge-Aware proceduraL text understAnding (KOALA) model, which effectively leverages multiple forms of external knowledge.
  arXiv  Detail & Related papers  (2020-09-28T10:28:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.