ShapeMOD: Macro Operation Discovery for 3D Shape Programs

• URL: http://arxiv.org/abs/2104.06392v1
• Date: Tue, 13 Apr 2021 17:54:03 GMT
• Title: ShapeMOD: Macro Operation Discovery for 3D Shape Programs
• Authors: R. Kenny Jones, David Charatan, Paul Guerrero, Niloy J. Mitra, Daniel Ritchie
• Abstract summary: We present ShapeMOD, an algorithm for automatically discovering macros that are useful across large datasets of 3D shape programs. ShapeMOD operates on shape programs expressed in an imperative, statement-based language. We show that it automatically discovers a concise set of macros that abstract out common structural and parametric patterns.
• Score: 41.973945628565104
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A popular way to create detailed yet easily controllable 3D shapes is via procedural modeling, i.e. generating geometry using programs. Such programs consist of a series of instructions along with their associated parameter values. To fully realize the benefits of this representation, a shape program should be compact and only expose degrees of freedom that allow for meaningful manipulation of output geometry. One way to achieve this goal is to design higher-level macro operators that, when executed, expand into a series of commands from the base shape modeling language. However, manually authoring such macros, much like shape programs themselves, is difficult and largely restricted to domain experts. In this paper, we present ShapeMOD, an algorithm for automatically discovering macros that are useful across large datasets of 3D shape programs. ShapeMOD operates on shape programs expressed in an imperative, statement-based language. It is designed to discover macros that make programs more compact by minimizing the number of function calls and free parameters required to represent an input shape collection. We run ShapeMOD on multiple collections of programs expressed in a domain-specific language for 3D shape structures. We show that it automatically discovers a concise set of macros that abstract out common structural and parametric patterns that generalize over large shape collections. We also demonstrate that the macros found by ShapeMOD improve performance on downstream tasks including shape generative modeling and inferring programs from point clouds. Finally, we conduct a user study that indicates that ShapeMOD's discovered macros make interactive shape editing more efficient.

Related papers

• ShapeLib: designing a library of procedural 3D shape abstractions with Large Language Models [46.372361125013974]
  We present ShapeLib, a library of 3D shape abstraction functions. We train a recognition network that learns to infer shape programs based on our library from different visual modalities. Our shape functions have parameters that are semantically interpretable and can be modified to produce plausible shape variations.
  arXiv  Detail & Related papers  (2025-02-13T01:52:02Z)
• CLIPtortionist: Zero-shot Text-driven Deformation for Manufactured 3D Shapes [15.985339563442189]
  We propose a zero-shot text-driven 3D shape deformation system that deforms an input 3D mesh of a manufactured object to fit an input text description. We find that CLIP-based objective functions exhibit many spurious local optima; to circumvent them, we parameterize deformations using a novel deformation model called BoxDefGraph.
  arXiv  Detail & Related papers  (2024-10-19T20:11:11Z)
• PyTorchGeoNodes: Enabling Differentiable Shape Programs for 3D Shape Reconstruction [27.98261111833689]
  We propose PyTorchGeoNodes, a differentiable module for reconstructing 3D objects from images using interpretable shape programs. In our experiments, we apply our algorithm to reconstruct 3D objects in the ScanNet dataset and evaluate our results against CAD model retrieval-based reconstructions.
  arXiv  Detail & Related papers  (2024-04-16T14:43:33Z)
• ShapeGPT: 3D Shape Generation with A Unified Multi-modal Language Model [27.122194733305594]
  We present ShapeGPT, a shape-included multi-modal framework to leverage strong pre-trained language models to address multiple shape-relevant tasks. Specifically, ShapeGPT employs a word-sentence-paragraph framework to discretize continuous shapes into shape words, further assembles these words for shape sentences, and integrates shape with instructional text for multi-modal paragraphs. Experiments demonstrate that ShapeGPT achieves comparable performance across shape-relevant tasks, including text-to-shape, shape-to-text, shape completion, and shape editing.
  arXiv  Detail & Related papers  (2023-11-29T13:26:29Z)
• Zero-Shot 3D Shape Correspondence [67.18775201037732]
  We propose a novel zero-shot approach to computing correspondences between 3D shapes. We exploit the exceptional reasoning capabilities of recent foundation models in language and vision. Our approach produces highly plausible results in a zero-shot manner, especially between strongly non-isometric shapes.
  arXiv  Detail & Related papers  (2023-06-05T21:14:23Z)
• Deep Implicit Templates for 3D Shape Representation [70.9789507686618]
  We propose a new 3D shape representation that supports explicit correspondence reasoning in deep implicit representations. Our key idea is to formulate DIFs as conditional deformations of a template implicit function. We show that our method can not only learn a common implicit template for a collection of shapes, but also establish dense correspondences across all the shapes simultaneously without any supervision.
  arXiv  Detail & Related papers  (2020-11-30T06:01:49Z)
• ShapeAssembly: Learning to Generate Programs for 3D Shape Structure Synthesis [38.27280837835169]
  We propose ShapeAssembly, a domain-specific "assembly-language" for 3D shape structures. We show how to extract ShapeAssembly programs from existing shape structures in the PartNet dataset. We evaluate our approach by comparing shapes output by our generated programs to those from other recent shape structure models.
  arXiv  Detail & Related papers  (2020-09-17T02:26:45Z)
• DSG-Net: Learning Disentangled Structure and Geometry for 3D Shape Generation [98.96086261213578]
  We introduce DSG-Net, a deep neural network that learns a disentangled structured and geometric mesh representation for 3D shapes. This supports a range of novel shape generation applications with disentangled control, such as of structure (geometry) while keeping geometry (structure) unchanged. Our method not only supports controllable generation applications but also produces high-quality synthesized shapes, outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-12T17:06:51Z)
• Unsupervised Learning of Intrinsic Structural Representation Points [50.92621061405056]
  Learning structures of 3D shapes is a fundamental problem in the field of computer graphics and geometry processing. We present a simple yet interpretable unsupervised method for learning a new structural representation in the form of 3D structure points.
  arXiv  Detail & Related papers  (2020-03-03T17:40:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.