ShapeLib: designing a library of procedural 3D shape abstractions with Large Language Models

• URL: http://arxiv.org/abs/2502.08884v1
• Date: Thu, 13 Feb 2025 01:52:02 GMT
• Title: ShapeLib: designing a library of procedural 3D shape abstractions with Large Language Models
• Authors: R. Kenny Jones, Paul Guerrero, Niloy J. Mitra, Daniel Ritchie,
• Abstract summary: We present ShapeLib, a library of 3D shape abstraction functions. We train a recognition network that learns to infer shape programs based on our library from different visual modalities. Our shape functions have parameters that are semantically interpretable and can be modified to produce plausible shape variations.
• Score: 46.372361125013974
• License:
• Abstract: Procedural representations are desirable, versatile, and popular shape encodings. Authoring them, either manually or using data-driven procedures, remains challenging, as a well-designed procedural representation should be compact, intuitive, and easy to manipulate. A long-standing problem in shape analysis studies how to discover a reusable library of procedural functions, with semantically aligned exposed parameters, that can explain an entire shape family. We present ShapeLib as the first method that leverages the priors of frontier LLMs to design a library of 3D shape abstraction functions. Our system accepts two forms of design intent: text descriptions of functions to include in the library and a seed set of exemplar shapes. We discover procedural abstractions that match this design intent by proposing, and then validating, function applications and implementations. The discovered shape functions in the library are not only expressive but also generalize beyond the seed set to a full family of shapes. We train a recognition network that learns to infer shape programs based on our library from different visual modalities (primitives, voxels, point clouds). Our shape functions have parameters that are semantically interpretable and can be modified to produce plausible shape variations. We show that this allows inferred programs to be successfully manipulated by an LLM given a text prompt. We evaluate ShapeLib on different datasets and show clear advantages over existing methods and alternative formulations.

Related papers

• ShapeCoder: Discovering Abstractions for Visual Programs from Unstructured Primitives [44.01940125080666]
  We present ShapeCoder, the first system capable of taking a dataset of shapes, represented with unstructured primitives. We show how ShapeCoder discovers a library of abstractions that capture high-level relationships, remove extraneous degrees of freedom, and achieve better dataset compression.
  arXiv  Detail & Related papers  (2023-05-09T17:55:48Z)
• PatchRD: Detail-Preserving Shape Completion by Learning Patch Retrieval and Deformation [59.70430570779819]
  We introduce a data-driven shape completion approach that focuses on completing geometric details of missing regions of 3D shapes. Our key insight is to copy and deform patches from the partial input to complete missing regions. We leverage repeating patterns by retrieving patches from the partial input, and learn global structural priors by using a neural network to guide the retrieval and deformation steps.
  arXiv  Detail & Related papers  (2022-07-24T18:59:09Z)
• Towards Implicit Text-Guided 3D Shape Generation [81.22491096132507]
  This work explores the challenging task of generating 3D shapes from text. We propose a new approach for text-guided 3D shape generation, capable of producing high-fidelity shapes with colors that match the given text description.
  arXiv  Detail & Related papers  (2022-03-28T10:20:03Z)
• ShapeMOD: Macro Operation Discovery for 3D Shape Programs [41.973945628565104]
  We present ShapeMOD, an algorithm for automatically discovering macros that are useful across large datasets of 3D shape programs. ShapeMOD operates on shape programs expressed in an imperative, statement-based language. We show that it automatically discovers a concise set of macros that abstract out common structural and parametric patterns.
  arXiv  Detail & Related papers  (2021-04-13T17:54:03Z)
• ShapeAssembly: Learning to Generate Programs for 3D Shape Structure Synthesis [38.27280837835169]
  We propose ShapeAssembly, a domain-specific "assembly-language" for 3D shape structures. We show how to extract ShapeAssembly programs from existing shape structures in the PartNet dataset. We evaluate our approach by comparing shapes output by our generated programs to those from other recent shape structure models.
  arXiv  Detail & Related papers  (2020-09-17T02:26:45Z)
• PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations [75.42959184226702]
  We present a new mid-level patch-based surface representation for object-agnostic training. We show several applications of our new representation, including shape and partial point cloud completion.
  arXiv  Detail & Related papers  (2020-08-04T15:34:46Z)
• Learning Generative Models of Shape Handles [43.41382075567803]
  We present a generative model to synthesize 3D shapes as sets of handles. Our model can generate handle sets with varying cardinality and different types of handles. We show that the resulting shape representations are intuitive and achieve superior quality than previous state-of-the-art.
  arXiv  Detail & Related papers  (2020-04-06T22:35:55Z)
• DualSDF: Semantic Shape Manipulation using a Two-Level Representation [54.62411904952258]
  We propose DualSDF, a representation expressing shapes at two levels of granularity, one capturing fine details and the other representing an abstracted proxy shape. Our two-level model gives rise to a new shape manipulation technique in which a user can interactively manipulate the coarse proxy shape and see the changes instantly mirrored in the high-resolution shape.
  arXiv  Detail & Related papers  (2020-04-06T17:59:15Z)
• Deep Geometric Functional Maps: Robust Feature Learning for Shape Correspondence [31.840880075039944]
  We present a novel learning-based approach for computing correspondences between non-rigid 3D shapes. Key to our method is a feature-extraction network that learns directly from raw shape geometry.
  arXiv  Detail & Related papers  (2020-03-31T15:20:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.