Learning Neural Deformation Representation for 4D Dynamic Shape Generation

• URL: http://arxiv.org/abs/2606.01021v1
• Date: Sun, 31 May 2026 05:25:11 GMT
• Title: Learning Neural Deformation Representation for 4D Dynamic Shape Generation
• Abstract summary: We focus on generating 4D dynamic shapes with an emphasis on both generation quality and efficiency.<n>We propose a new neural deformation representation and combine it with conditional neural signed distance fields to design a 4D representation architecture.<n>The proposed deformation representation, which works by predicting skinning weights and rigid transformations for multiple parts, also has advantages over the deformation modules of existing 4D representations.
• Score: 18.39851711911021
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent developments in 3D shape representation opened new possibilities for generating detailed 3D shapes. Despite these advances, there are few studies dealing with the generation of 4D dynamic shapes that have the form of 3D objects deforming over time. To bridge this gap, we focus on generating 4D dynamic shapes with an emphasis on both generation quality and efficiency in this paper. HyperDiffusion, a previous work on 4D generation, proposed a method of directly generating the weight parameters of 4D occupancy fields but suffered from low temporal consistency and slow rendering speed due to motion representation that is not separated from the shape representation of 4D occupancy fields. Therefore, we propose a new neural deformation representation and combine it with conditional neural signed distance fields to design a 4D representation architecture in which the motion latent space is disentangled from the shape latent space. The proposed deformation representation, which works by predicting skinning weights and rigid transformations for multiple parts, also has advantages over the deformation modules of existing 4D representations in understanding the structure of shapes. In addition, we design a training process of a diffusion model that utilizes the shape and motion features that are extracted by our 4D representation as data points. The results of unconditional generation, conditional generation, and motion retargeting experiments demonstrate that our method not only shows better performance than previous works in 4D dynamic shape generation but also has various potential applications.

Related papers

• Sculpt4D: Generating 4D Shapes via Sparse-Attention Diffusion Transformers [70.55535980817577]
  Sculpt4D is a native 4D generative framework that seamlessly integrates efficient temporal modeling into a pretrained 3D Transformer.<n>We show that Sculpt4D establishes a new state-of-the-art in temporally coherent 4D synthesis and charts a path toward efficient and scalable 4D generation.
  arXiv  Detail & Related papers  (2026-04-23T12:18:55Z)
• DNF: Unconditional 4D Generation with Dictionary-based Neural Fields [32.25164913000621]
  We propose DNF, a new 4D representation for unconditional generative modeling.<n>D DNF efficiently models deformable shapes with disentangled shape and motion.<n>Our method is able to generate effective, high-fidelity 4D animations.
  arXiv  Detail & Related papers  (2024-12-06T16:25:57Z)
• Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Models [116.31344506738816]
  We present a novel framework, textbfDiffusion4D, for efficient and scalable 4D content generation. We develop a 4D-aware video diffusion model capable of synthesizing orbital views of dynamic 3D assets. Our method surpasses prior state-of-the-art techniques in terms of generation efficiency and 4D geometry consistency.
  arXiv  Detail & Related papers  (2024-05-26T17:47:34Z)
• SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer [57.506654943449796]
  We propose an efficient, sparse-controlled video-to-4D framework named SC4D that decouples motion and appearance. Our method surpasses existing methods in both quality and efficiency. We devise a novel application that seamlessly transfers motion onto a diverse array of 4D entities.
  arXiv  Detail & Related papers  (2024-04-04T18:05:18Z)
• Learning Coherent Matrixized Representation in Latent Space for Volumetric 4D Generation [48.671462912294594]
  We propose a framework that generates 4D sequences with dynamic evolution in shape and color across spatial and temporal dimensions.<n>We first use a coherent 3D shape and color modeling to encode shape and color each detailed 3D geometry frame into a latent space.<n>Then we propose a matrixized 4D sequence representation allowing efficient diffusion model operation.
  arXiv  Detail & Related papers  (2024-03-20T01:59:43Z)
• LoRD: Local 4D Implicit Representation for High-Fidelity Dynamic Human Modeling [69.56581851211841]
  We propose a novel Local 4D implicit Representation for Dynamic clothed human, named LoRD. Our key insight is to encourage the network to learn the latent codes of local part-level representation. LoRD has strong capability for representing 4D human, and outperforms state-of-the-art methods on practical applications.
  arXiv  Detail & Related papers  (2022-08-18T03:49:44Z)
• H4D: Human 4D Modeling by Learning Neural Compositional Representation [75.34798886466311]
  This work presents a novel framework that can effectively learn a compact and compositional representation for dynamic human. A simple yet effective linear motion model is proposed to provide a rough and regularized motion estimation. Experiments demonstrate our method is not only efficacy in recovering dynamic human with accurate motion and detailed geometry, but also amenable to various 4D human related tasks.
  arXiv  Detail & Related papers  (2022-03-02T17:10:49Z)
• Learning Parallel Dense Correspondence from Spatio-Temporal Descriptors for Efficient and Robust 4D Reconstruction [43.60322886598972]
  This paper focuses on the task of 4D shape reconstruction from a sequence of point clouds. We present a novel pipeline to learn a temporal evolution of the 3D human shape through capturing continuous transformation functions among cross-frame occupancy fields.
  arXiv  Detail & Related papers  (2021-03-30T13:36:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.