DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulation

• URL: http://arxiv.org/abs/2311.12194v2
• Date: Fri, 29 Mar 2024 19:37:18 GMT
• Title: DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulation
• Authors: Yifei Li, Hsiao-yu Chen, Egor Larionov, Nikolaos Sarafianos, Wojciech Matusik, Tuur Stuyck,
• Abstract summary: Physical simulations can produce realistic motions for clothed humans, but they require high-quality garment assets with associated physical parameters for cloth simulations. We propose papername,a novel approach that performs body and garment co-optimization using differentiable simulation. Our experiments demonstrate that our approach generates realistic clothing and body shape suitable for downstream applications.
• Score: 27.553678582454648
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The realism of digital avatars is crucial in enabling telepresence applications with self-expression and customization. While physical simulations can produce realistic motions for clothed humans, they require high-quality garment assets with associated physical parameters for cloth simulations. However, manually creating these assets and calibrating their parameters is labor-intensive and requires specialized expertise. Current methods focus on reconstructing geometry, but don't generate complete assets for physics-based applications. To address this gap, we propose \papername,~a novel approach that performs body and garment co-optimization using differentiable simulation. By integrating physical simulation into the optimization loop and accounting for the complex nonlinear behavior of cloth and its intricate interaction with the body, our framework recovers body and garment geometry and extracts important material parameters in a physically plausible way. Our experiments demonstrate that our approach generates realistic clothing and body shape suitable for downstream applications. We provide additional insights and results on our webpage: https://people.csail.mit.edu/liyifei/publication/diffavatar/

Related papers

• A Physics-embedded Deep Learning Framework for Cloth Simulation [6.8806198396336935]
  This paper proposes a physics-embedded learning framework that directly encodes physical features of cloth simulation. The framework can also integrate with other external forces and collision handling through either traditional simulators or sub neural networks.
  arXiv  Detail & Related papers  (2024-03-19T15:21:00Z)
• AniDress: Animatable Loose-Dressed Avatar from Sparse Views Using Garment Rigging Model [58.035758145894846]
  We introduce AniDress, a novel method for generating animatable human avatars in loose clothes using very sparse multi-view videos. A pose-driven deformable neural radiance field conditioned on both body and garment motions is introduced, providing explicit control of both parts. Our method is able to render natural garment dynamics that deviate highly from the body and well to generalize to both unseen views and poses.
  arXiv  Detail & Related papers  (2024-01-27T08:48:18Z)
• DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models [102.13968267347553]
  We present DiffuseBot, a physics-augmented diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks. We showcase a range of simulated and fabricated robots along with their capabilities.
  arXiv  Detail & Related papers  (2023-11-28T18:58:48Z)
• Benchmarking the Sim-to-Real Gap in Cloth Manipulation [10.530012817995656]
  We present a benchmark dataset to evaluate the sim-to-real gap in cloth manipulation. We use the dataset to evaluate the reality gap, computational time, and stability of four popular deformable object simulators.
  arXiv  Detail & Related papers  (2023-10-14T09:36:01Z)
• Towards Multi-Layered 3D Garments Animation [135.77656965678196]
  Existing approaches mostly focus on single-layered garments driven by only human bodies and struggle to handle general scenarios. We propose a novel data-driven method, called LayersNet, to model garment-level animations as particle-wise interactions in a micro physics system. Our experiments show that LayersNet achieves superior performance both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2023-05-17T17:53:04Z)
• Near-realtime Facial Animation by Deep 3D Simulation Super-Resolution [7.14576106770047]
  We present a neural network-based simulation framework that can efficiently and realistically enhance a facial performance produced by a low-cost, realtime physics-based simulation. We use face animation as an exemplar of such a simulation domain, where creating this semantic congruence is achieved by simply dialing in the same muscle actuation controls and skeletal pose in the two simulators. Our proposed neural network super-resolution framework generalizes from this training set to unseen expressions, compensates for modeling discrepancies between the two simulations due to limited resolution or cost-cutting approximations in the real-time variant, and does not require any semantic descriptors or parameters to
  arXiv  Detail & Related papers  (2023-05-05T00:09:24Z)
• Dressing Avatars: Deep Photorealistic Appearance for Physically Simulated Clothing [49.96406805006839]
  We introduce pose-driven avatars with explicit modeling of clothing that exhibit both realistic clothing dynamics and photorealistic appearance learned from real-world data. Our key contribution is a physically-inspired appearance network, capable of generating photorealistic appearance with view-dependent and dynamic shadowing effects even for unseen body-clothing configurations.
  arXiv  Detail & Related papers  (2022-06-30T17:58:20Z)
• Differentiable Dynamics for Articulated 3d Human Motion Reconstruction [29.683633237503116]
  We introduce DiffPhy, a differentiable physics-based model for articulated 3d human motion reconstruction from video. We validate the model by demonstrating that it can accurately reconstruct physically plausible 3d human motion from monocular video.
  arXiv  Detail & Related papers  (2022-05-24T17:58:37Z)
• Inferring Articulated Rigid Body Dynamics from RGBD Video [18.154013621342266]
  We introduce a pipeline that combines inverse rendering with differentiable simulation to create digital twins of real-world articulated mechanisms. Our approach accurately reconstructs the kinematic tree of an articulated mechanism being manipulated by a robot.
  arXiv  Detail & Related papers  (2022-03-20T08:19:02Z)
• GeoSim: Photorealistic Image Simulation with Geometry-Aware Composition [81.24107630746508]
  We present GeoSim, a geometry-aware image composition process that synthesizes novel urban driving scenes. We first build a diverse bank of 3D objects with both realistic geometry and appearance from sensor data. The resulting synthetic images are photorealistic, traffic-aware, and geometrically consistent, allowing image simulation to scale to complex use cases.
  arXiv  Detail & Related papers  (2021-01-16T23:00:33Z)
• Cloth in the Wind: A Case Study of Physical Measurement through Simulation [50.31424339972478]
  We propose to measure latent physical properties for cloth in the wind without ever having seen a real example before. Our solution is an iterative refinement procedure with simulation at its core. The correspondence is measured using an embedding function that maps physically similar examples to nearby points.
  arXiv  Detail & Related papers  (2020-03-09T21:32:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.