Related papers: Shape Generation via Weight Space Learning

Shape Generation via Weight Space Learning

URL: http://arxiv.org/abs/2503.21830v1
Date: Wed, 26 Mar 2025 15:49:27 GMT
Title: Shape Generation via Weight Space Learning
Authors: Maximilian Plattner, Arturs Berzins, Johannes Brandstetter,
Abstract summary: We show that submanifolds within a large 3D shape-generative model can modulate topological properties or fine-grained part features separately.<n>Results highlight the potential of weight space learning to unlock new approaches for 3D shape generation and specialized fine-tuning.
Score: 12.429026910048528
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Foundation models for 3D shape generation have recently shown a remarkable capacity to encode rich geometric priors across both global and local dimensions. However, leveraging these priors for downstream tasks can be challenging as real-world data are often scarce or noisy, and traditional fine-tuning can lead to catastrophic forgetting. In this work, we treat the weight space of a large 3D shape-generative model as a data modality that can be explored directly. We hypothesize that submanifolds within this high-dimensional weight space can modulate topological properties or fine-grained part features separately, demonstrating early-stage evidence via two experiments. First, we observe a sharp phase transition in global connectivity when interpolating in conditioning space, suggesting that small changes in weight space can drastically alter topology. Second, we show that low-dimensional reparameterizations yield controlled local geometry changes even with very limited data. These results highlight the potential of weight space learning to unlock new approaches for 3D shape generation and specialized fine-tuning.

Related papers

Geometric Trajectory Diffusion Models [58.853975433383326]
Generative models have shown great promise in generating 3D geometric systems. Existing approaches only operate on static structures, neglecting the fact that physical systems are always dynamic in nature. We propose geometric trajectory diffusion models (GeoTDM), the first diffusion model for modeling the temporal distribution of 3D geometric trajectories.
arXiv Detail & Related papers (2024-10-16T20:36:41Z)
Maintaining Structural Integrity in Parameter Spaces for Parameter Efficient Fine-tuning [78.39310274926535]
Adapting pre-trained foundation models for various downstream tasks has been prevalent in artificial intelligence.<n>To mitigate this, several fine-tuning techniques have been developed to update the pre-trained model weights in a more resource-efficient manner.<n>This paper introduces a generalized parameter-efficient fine-tuning framework, designed for various dimensional parameter space.
arXiv Detail & Related papers (2024-05-23T16:04:42Z)
Non-rigid Structure-from-Motion: Temporally-smooth Procrustean Alignment and Spatially-variant Deformation Modeling [34.606331252248886]
Non-rigid Structure-from-Motion (NRSfM) has been extensively studied and great progress has been made. There are still key challenges that hinder their broad real-world applications. This paper proposes to resolve the above issues from a spatial-temporal modeling perspective.
arXiv Detail & Related papers (2024-05-07T13:33:50Z)
GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image [94.56927147492738]
We introduce GeoWizard, a new generative foundation model designed for estimating geometric attributes from single images. We show that leveraging diffusion priors can markedly improve generalization, detail preservation, and efficiency in resource usage. We propose a simple yet effective strategy to segregate the complex data distribution of various scenes into distinct sub-distributions.
arXiv Detail & Related papers (2024-03-18T17:50:41Z)
Explorable Mesh Deformation Subspaces from Unstructured Generative Models [53.23510438769862]
Deep generative models of 3D shapes often feature continuous latent spaces that can be used to explore potential variations. We present a method to explore variations among a given set of landmark shapes by constructing a mapping from an easily-navigable 2D exploration space to a subspace of a pre-trained generative model.
arXiv Detail & Related papers (2023-10-11T18:53:57Z)
SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements [62.652588951757764]
Learning to model and reconstruct humans in clothing is challenging due to articulation, non-rigid deformation, and varying clothing types and topologies. Recent work uses neural networks to parameterize local surface elements. We present three key innovations: First, we deform surface elements based on a human body model. Second, we address the limitations of existing neural surface elements by regressing local geometry from local features.
arXiv Detail & Related papers (2021-04-15T17:59:39Z)
Disentangling Geometric Deformation Spaces in Generative Latent Shape Models [5.582957809895198]
A complete representation of 3D objects requires characterizing the space of deformations in an interpretable manner. We improve on a prior generative model of disentanglement for 3D shapes, wherein the space of object geometry is factorized into rigid orientation, non-rigid pose, and intrinsic shape. The resulting model can be trained from raw 3D shapes, without correspondences, labels, or even rigid alignment.
arXiv Detail & Related papers (2021-02-27T06:54:31Z)
Dense Non-Rigid Structure from Motion: A Manifold Viewpoint [162.88686222340962]
Non-Rigid Structure-from-Motion (NRSfM) problem aims to recover 3D geometry of a deforming object from its 2D feature correspondences across multiple frames. We show that our approach significantly improves accuracy, scalability, and robustness against noise.
arXiv Detail & Related papers (2020-06-15T09:15:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.