InstructMesh: Selective Refinement of Generative 3D Models for Fabrication

• URL: http://arxiv.org/abs/2608.28534v1
• Date: Fri, 28 Aug 2026 17:09:42 GMT
• Title: InstructMesh: Selective Refinement of Generative 3D Models for Fabrication
• Abstract summary: We present InstructMesh, a tool that enables selective repair of generative 3D models through region selection and targeted operations.<n>We first analyze common fabrication-related failure modes in outputs from state-of-the-art generative tools.<n>We then conduct two user studies, demonstrating that novices can identify and perform fabrication-relevant repairs on generative outputs using InstructMesh.
• Score: 58.233035582921474
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in generative AI allow users to create 3D models from text or images. However, these models prioritize visual plausibility over geometric accuracy, often generating results with flaws that compromise their intended use post-fabrication. We present InstructMesh, an interactive post-generation refinement tool that enables selective repair of generative 3D models through region selection and targeted operations, such as opening or sealing voids, or adjusting local thickness. Users can invoke edit operations via natural language prompts or slider controls. By operating directly on the intermediate latent representation, InstructMesh allows users to apply robust geometric corrections without requiring expert modeling skills. To inform our design, we first analyze common fabrication-related failure modes in outputs from state-of-the-art generative tools. We then conduct two user studies, demonstrating that novices can identify and perform fabrication-relevant repairs on generative outputs using InstructMesh, and revealing user preference for hybrid interfaces that combine slider controls with natural language input.

Related papers

• High-Fidelity 3D Facial Avatar Synthesis with Controllable Fine-Grained Expressions [83.73866089033585]
  We propose a novel approach by simultaneously refining both the latent code of a pretrained 3D-Aware GAN model for texture editing and the expression code of the driven 3DMM model for mesh editing.<n>To optimize the Dual Mappers, we propose a Text-Guided Optimization method, leveraging a CLIP-based objective function with expression text prompts as targets.
  arXiv  Detail & Related papers  (2026-03-16T03:25:10Z)
• Proc3D: Procedural 3D Generation and Parametric Editing of 3D Shapes with Large Language Models [21.349049326631867]
  Proc3D is a system designed to generate editable 3D models while enabling real-time modifications.<n>At its core, Proc3D introduces procedural compact graph (PCG), a graph representation of 3D models.<n>We demonstrate Proc3D's capabilities using two generative approaches: GPT-4o with in-context learning (ICL) and a fine-tuned LLAMA-3 model.
  arXiv  Detail & Related papers  (2026-01-18T03:08:08Z)
• StdGEN++: A Comprehensive System for Semantic-Decomposed 3D Character Generation [57.06461272772509]
  StdGEN++ is a novel and comprehensive system for generating high-fidelity, semantically decomposed 3D characters from diverse inputs.<n>It achieves state-of-the-art performance, significantly outperforming existing methods in geometric accuracy and semantic disentanglement.<n>The resulting structural independence unlocks advanced downstream capabilities, including non-destructive editing, physics-compliant animation, and gaze tracking.
  arXiv  Detail & Related papers  (2026-01-12T15:41:27Z)
• SpaceControl: Introducing Test-Time Spatial Control to 3D Generative Modeling [62.89824987879374]
  We introduce SpaceControl, a training-free test-time method for explicit spatial control of 3D generation.<n>SpaceControl integrates seamlessly with modern pre-trained generative models without requiring any additional training.<n>We present an interactive user interface that enables online editing of superquadrics for direct conversion into textured 3D assets.
  arXiv  Detail & Related papers  (2025-12-05T00:54:48Z)
• INST-Sculpt: Interactive Stroke-based Neural SDF Sculpting [4.398358164188111]
  We introduce a framework that enables interactive surface sculpting edits directly on neural implicit representations.<n>Our approach allows users to perform stroke-based modifications on the fly, ensuring intuitive shape manipulation without switching representations.
  arXiv  Detail & Related papers  (2025-02-05T05:25:15Z)
• Coin3D: Controllable and Interactive 3D Assets Generation with Proxy-Guided Conditioning [52.81032340916171]
  Coin3D allows users to control the 3D generation using a coarse geometry proxy assembled from basic shapes. Our method achieves superior controllability and flexibility in the 3D assets generation task.
  arXiv  Detail & Related papers  (2024-05-13T17:56:13Z)
• Interactive3D: Create What You Want by Interactive 3D Generation [13.003964182554572]
  We introduce Interactive3D, an innovative framework for interactive 3D generation that grants users precise control over the generative process. Our experiments demonstrate that Interactive3D markedly improves the controllability and quality of 3D generation.
  arXiv  Detail & Related papers  (2024-04-25T11:06:57Z)
• 3D-GPT: Procedural 3D Modeling with Large Language Models [47.72968643115063]
  We introduce 3D-GPT, a framework utilizing large language models(LLMs) for instruction-driven 3D modeling. 3D-GPT positions LLMs as proficient problem solvers, dissecting the procedural 3D modeling tasks into accessible segments and appointing the apt agent for each task. Our empirical investigations confirm that 3D-GPT not only interprets and executes instructions, delivering reliable results but also collaborates effectively with human designers.
  arXiv  Detail & Related papers  (2023-10-19T17:41:48Z)
• 3D Generative Model Latent Disentanglement via Local Eigenprojection [13.713373496487012]
  We introduce a novel loss function grounded in spectral geometry for different neural-network-based generative models of 3D head and body meshes. Experimental results show that our local eigenprojection disentangled (LED) models offer improved disentanglement with respect to the state-of-the-art.
  arXiv  Detail & Related papers  (2023-02-24T18:19:49Z)
• Training and Tuning Generative Neural Radiance Fields for Attribute-Conditional 3D-Aware Face Generation [66.21121745446345]
  We propose a conditional GNeRF model that integrates specific attribute labels as input, thus amplifying the controllability and disentanglement capabilities of 3D-aware generative models. Our approach builds upon a pre-trained 3D-aware face model, and we introduce a Training as Init and fidelity for Tuning (TRIOT) method to train a conditional normalized flow module. Our experiments substantiate the efficacy of our model, showcasing its ability to generate high-quality edits with enhanced view consistency.
  arXiv  Detail & Related papers  (2022-08-26T10:05:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.